在并行 bash 脚本中运行多个脚本

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/15644991/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-09 23:29:56  来源:igfitidea点击:

Running several scripts in parallel bash script

bashparallel-processing

提问by Bdar

I have a bash script that contains other scripts inside that are run in series. However, it takes a decent amount of time to run them all. Is there a way to run these scripts in parallel to improve overall perfomance? They are independent of each other.

我有一个 bash 脚本,其中包含串行运行的其他脚本。但是,运行它们需要相当长的时间。有没有办法并行运行这些脚本来提高整体性能?它们彼此独立。

It looks similar to:

它看起来类似于:

#!/bin/bash

#some code here
cppcheck.sh
churn.sh
run.sh


Update:

更新:

**git log --pretty=format: --numstat | perl -ane'$c{$F[2]} += abs($F[0]+$F[1]) 
if $F[2];END {print "$_\t$c{$_}\n" for sort keys %c}' > ${OUTPUT_DIR}/churn.txt**
sed -i -e '/deps/d;/build/d;/translations/d;/tests/d' -e 30q ${OUTPUT_DIR}/churn.txt
sort -r -n -t$'\t' -k2 ${OUTPUT_DIR}/churn.txt -o ${OUTPUT_DIR}/churn.txt
echo "set term canvas size 1200, 800; set output '${OUTPUT_DIR}/output.html'; 
unset key; set bmargin at screen 0.4; set xtics rotate by -90 scale 0,0; 
set ylabel 'Number of lines changed (total)'; set title 'Files with high churn 
level';set boxwidth 0.7; set style fill solid; set noborder; 
plot '${OUTPUT_DIR}/churn.txt' using 2:xticlabels(1) with boxes" | gnuplot
echo "finished running churn.sh!"

This is the code inside churn.sh. The bold command takes 40 or so secs to implement. If in a main script I put ampersand after churn.sh &, it throws an error saying sed can't read churn.txt file (since it's not created yet). It seems that it doesn't wait till the output is saved in a file. I inserted wait after that command but it doesn't help.

这是 churn.sh 中的代码。粗体命令需要 40 秒左右的时间来实现。如果在主脚本中,我在 churn.sh & 后面放了 &,它会抛出一个错误,说 sed 无法读取 churn.txt 文件(因为它尚未创建)。似乎它不会等到输出保存在文件中。我在该命令之后插入了等待,但它没有帮助。

回答by Anirudh Ramanathan

Using the &to run it in the background will do the trick

使用&在后台运行它就可以了

cppcheck.sh &
churn.sh &
run.sh &

wait
echo "All 3 complete"

It will fork off a new process for each of them.

它将为每个人分叉一个新进程。

The bash waitwill also come in handy as stated in the comments, if you have something to be run on the parent script, afterthese three finish.

wait评论中所述,bash也将派上用场,如果您要在父脚本上运行某些内容,则这三个完成后。

Without an argument it will wait for all child processes to complete, and then resume execution of the parent script.

如果没有参数,它将等待所有子进程完成,然后恢复父脚本的执行。



The issues you are facing seem to be directly related to this. Variables set are only visible to the sub-shell in which they are defined. So, if you have OUT_DIRspecified in the parent script, it won't be visible to the child script when it forks off. The right thing to do in this case would be to exportthe variable as an environment variable.

您面临的问题似乎与此直接相关。变量集仅对定义它们子 shell可见。因此,如果您已OUT_DIR在父脚本中指定,则子脚本在分叉时将看不到它。在这种情况下,正确的做法是export将变量作为环境变量。