bash 限制大小 wget 可以下载

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/20317945/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-18 08:49:32  来源:igfitidea点击:

Limit size wget can download

bashunixwget

提问by d-_-b

Is it possible to limit or cap the amount of data wgetdownloads from a site? Either via server setting or wgetsetting?

是否可以限制或限制wget从站点下载的数据量?通过服务器设置还是wget设置?

For example, one page is 1GB size, I want wget to stop downloading at 100MB.

例如,一页大小为 1GB,我希望 wget 在 100MB 时停止下载。

回答by jlliagre

Leveraging the ability for the system to limit processes resource consumption through the ulimitcommand should just work. Assuming you use bash:

利用系统通过ulimit命令限制进程资源消耗的能力应该可以正常工作。假设您使用 bash:

(ulimit -f 102400; wget $url)

Here the size is in 1024 bytes blocks. Note that if you use a different still standard compliant shell, or use bashin POSIX mode, the block size is 512 bytes so the command should be:

这里的大小以 1024 字节块为单位。请注意,如果您使用不同的仍然符合标准的 shell,或bash在 POSIX 模式下使用,则块大小为 512 字节,因此命令应为:

(ulimit -f 204800; wget $url)

回答by janos

See the -Qor --quotaoptions in man wget.

请参阅 中的-Q--quota选项man wget

Specify download quota for automatic retrievals. The value can be specified in bytes (default), kilobytes (with k suffix), or megabytes (with m suffix).

Note that quota will never affect downloading a single file. So if you specify wget -Q10k ftp://wuarchive.wustl.edu/ls-lR.gz, all of the ls-lR.gz will be downloaded. The same goes even when several URLs are specified on the command-line. However, quota is respected when retrieving either recursively, or from an input file. Thus you may safely type wget -Q2m -i sites---download will be aborted when the quota is exceeded.

指定自动检索的下载配额。该值可以以字节(默认)、千字节(带有 k 后缀)或兆字节(带有 m 后缀)为单位指定。

请注意,配额永远不会影响下载单个文件。因此,如果您指定 wget -Q10k ftp://wuarchive.wustl.edu/ls-lR.gz,则将下载所有 ls-lR.gz 。即使在命令行上指定了多个 URL,情况也是如此。但是,在以递归方式或从输入文件中检索时,会遵守配额。因此您可以安全地输入 wget -Q2m -i sites---当超过配额时下载将被中止。

Unfortunately, as the text explains, this might not be useful for you: if you're trying to download a specific page or file, this limit won't apply. The quota will apply only when downloading something recursively.

不幸的是,正如文本所解释的,这可能对您没有用:如果您尝试下载特定页面或文件,则此限制将不适用。配额仅在递归下载某些内容时适用。

Another option is to run the download in the background, monitor the filesize and kill wgetwhen the file grows bigger than the limit.

另一种选择是在后台运行下载,监控文件大小并wget在文件增长超过限制时终止。

回答by russellm