bash 如何在一个命令中卷曲多个资源?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/17447746/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-18 05:49:48  来源:igfitidea点击:

How do I curl multiple resources in one command?

bashcurl

提问by Coder

Say I am trying to download a set of 50 lecture notes efficiently. These notes are inside the profsubdirectory of a university website. The 45th lecture note is inside the lect45subdirectory as a pdf entitled lect45.pdf. I get my first pdf as follows:

假设我正在尝试有效地下载一组 50 个讲义。这些笔记prof位于大学网站的子目录中。第 45 个讲义位于lect45子目录中,名为 .pdf 的 pdf 文件lect45.pdf。我得到我的第一个 pdf 如下:

curl -O http://www.university.edu/~prof/lect1/lect1.pdf

How do I get all my 50 notes efficiently using cURL and bash? I'm trying to do this from the command line, not through a Python / Ruby / Perl script. I know something like the below will generate a lot of 404s:

如何使用 cURL 和 bash 有效地获取所有 50 个笔记?我正在尝试从命令行执行此操作,而不是通过 Python/Ruby/Perl 脚本。我知道类似下面的内容会产生很多 404:

curl -O http://www.university.edu/~prof/lect{1..50}/lect{1..50}.pdf

so what will work better? I would prefer an elegant one-liner over a loop.

那么什么会更好呢?我更喜欢优雅的单线而不是循环。

回答by Alfe

Do it in several processes:

分几个过程做:

for i in {1..50}
do
    curl -O http://www.university.edu/~prof/lect$i/lect$i.pdf &
done

or as a one-liner (just a different formatting):

或作为单行(只是不同的格式):

for i in {1..50}; do curl -O http://www.university.edu/~prof/lect$i/lect$i.pdf & done

The &makes all processes run in parallel.

&使得所有进程并行运行。

Don't be scared by the output; the shell tells you that 50 processes have been started, that's a lot of spam. Later it will tell you for each of these that they terminated. A lot of output again.

不要被输出吓到;shell 告诉您已经启动了 50 个进程,这是很多垃圾邮件。稍后它会告诉你他们终止的每一个。再次大量输出。

You probably don't want to run all 50 in parallel ;-)

您可能不想并行运行所有 50 个 ;-)

EDIT:

编辑:

Your example using {1..50}twice makes a matrix of the numbers. See for example echo {1..3}/{1..3}to see what I mean. And I guess that this way you create a lot of 404s.

您使用{1..50}两次的示例构成了数字矩阵。例如echo {1..3}/{1..3},请参阅以了解我的意思。我猜这样你会创建很多 404。

回答by ДМИТРИЙ МАЛИКОВ

Take a look at parallelshell tool.

看看parallelshell工具。

So, for this particular case it will look like

所以,对于这种特殊情况,它看起来像

seq 50 | parallel curl -O http://www.university.edu/~prof/lect{}/lect{}.pdf 

As for curl- it doesn't have its own parallel mechanism, and what for it actually should? And your example with shell expansions {1..50}seems valid for me.

至于curl- 它没有自己的并行机制,它实际上应该做什么?你的 shell 扩展示例{1..50}对我来说似乎有效。