在 bash 中进行并行处理？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/3811943/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Doing parallel processing in bash?
提问by mark
I've thousands of png files which I like to make smaller with pngcrush. I've a simple find .. -execjob, but it's sequential. My machine has quite some resources and I'd make this in parallel.
我有数千个 png 文件，我喜欢用pngcrush. 我有一份简单的find .. -exec工作，但它是连续的。我的机器有相当多的资源，我会并行进行。
The operation to be performed on every png is:
要对每个 png 执行的操作是：
pngcrush input output && mv output input
Ideally I can specify the maximum number of parallel operations.
理想情况下，我可以指定最大并行操作数。
Is there a way to do this with bash and/or other shell helpers? I'm Ubuntu or Debian.
有没有办法用 bash 和/或其他 shell 助手来做到这一点？我是 Ubuntu 或 Debian。
回答by Bart Sas
You can use xargsto run multiple processes in parallel:
您可以使用xargs并行运行多个进程：
find /path -print0 | xargs -0 -n 1 -P <nr_procs> sh -c 'pngcrush temp.$$ && mv temp.$$ ' sh
xargswill read the list of files produced by find (separated by 0 characters (-0)) and run the provided command (sh -c '...' sh) with one parameter at a time (-n 1). xargs will run <nr_procs>(-P <nr_procs>) in parallel.
xargs将读取 find 生成的文件列表（以 0 个字符 ( -0)分隔）并一次sh -c '...' sh使用一个参数( )运行提供的命令( -n 1)。xargs 将并行运行<nr_procs>( -P <nr_procs>)。
回答by tokland
You can use custom find/xargssolutions (see Bart Sas' answer), but when things become more complex you have -at least- two powerful options:
您可以使用自定义find/xargs解决方案（请参阅 Bart Sas 的回答），但是当事情变得更加复杂时，您至少有两个强大的选择：
parallel(from package moreutils)- GNU parallel
回答by Ole Tange
With GNU Parallel http://www.gnu.org/software/parallel/it can be done like:
使用 GNU Parallel http://www.gnu.org/software/parallel/可以这样做：
find /path -print0 | parallel -0 pngcrush {} {.}.temp '&&' mv {.}.temp {}
Learn more:
了解更多：
- Watch the intro video for a quick introduction: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
- Walk through the tutorial (man parallel_tutorial). You command line will love you for it.
- 观看介绍视频以进行快速介绍：https: //www.youtube.com/playlist?list=PL284C9FF2488BC6D1
- 完成教程 (man parallel_tutorial)。您的命令行会因此而爱您。
