bash 限制大小 wget 可以下载
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/20317945/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Limit size wget can download
提问by d-_-b
Is it possible to limit or cap the amount of data wget
downloads from a site? Either via server setting or wget
setting?
是否可以限制或限制wget
从站点下载的数据量?通过服务器设置还是wget
设置?
For example, one page is 1GB size, I want wget to stop downloading at 100MB.
例如,一页大小为 1GB,我希望 wget 在 100MB 时停止下载。
回答by jlliagre
Leveraging the ability for the system to limit processes resource consumption through the ulimit
command should just work. Assuming you use bash:
利用系统通过ulimit
命令限制进程资源消耗的能力应该可以正常工作。假设您使用 bash:
(ulimit -f 102400; wget $url)
Here the size is in 1024 bytes blocks. Note that if you use a different still standard compliant shell, or use bash
in POSIX mode, the block size is 512 bytes so the command should be:
这里的大小以 1024 字节块为单位。请注意,如果您使用不同的仍然符合标准的 shell,或bash
在 POSIX 模式下使用,则块大小为 512 字节,因此命令应为:
(ulimit -f 204800; wget $url)
回答by janos
See the -Q
or --quota
options in man wget
.
请参阅 中的-Q
或--quota
选项man wget
。
Specify download quota for automatic retrievals. The value can be specified in bytes (default), kilobytes (with k suffix), or megabytes (with m suffix).
Note that quota will never affect downloading a single file. So if you specify wget -Q10k ftp://wuarchive.wustl.edu/ls-lR.gz, all of the ls-lR.gz will be downloaded. The same goes even when several URLs are specified on the command-line. However, quota is respected when retrieving either recursively, or from an input file. Thus you may safely type wget -Q2m -i sites---download will be aborted when the quota is exceeded.
指定自动检索的下载配额。该值可以以字节(默认)、千字节(带有 k 后缀)或兆字节(带有 m 后缀)为单位指定。
请注意,配额永远不会影响下载单个文件。因此,如果您指定 wget -Q10k ftp://wuarchive.wustl.edu/ls-lR.gz,则将下载所有 ls-lR.gz 。即使在命令行上指定了多个 URL,情况也是如此。但是,在以递归方式或从输入文件中检索时,会遵守配额。因此您可以安全地输入 wget -Q2m -i sites---当超过配额时下载将被中止。
Unfortunately, as the text explains, this might not be useful for you: if you're trying to download a specific page or file, this limit won't apply. The quota will apply only when downloading something recursively.
不幸的是,正如文本所解释的,这可能对您没有用:如果您尝试下载特定页面或文件,则此限制将不适用。配额仅在递归下载某些内容时适用。
Another option is to run the download in the background, monitor the filesize and kill wget
when the file grows bigger than the limit.
另一种选择是在后台运行下载,监控文件大小并wget
在文件增长超过限制时终止。
回答by russellm
Check here for a discussion, the first link in a google search:
在这里查看讨论,谷歌搜索中的第一个链接: