bash 如何限制bash函数中使用的线程/子进程数
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/6511884/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to limit number of threads/sub-processes used in a function in bash
提问by mentatkgs
My question is how change this code so it will use only 4 threads/sub-processes?
我的问题是如何更改此代码使其仅使用 4 个线程/子进程？
TESTS="a b c d e"
for f in $TESTS; do
t=$[ ( $RANDOM % 5 ) + 1 ]
sleep $t && echo $f $t &
done
wait
回答by Lynch
Interesting question. I tried to use xargs for this and I found a way.
有趣的问题。我尝试为此使用 xargs 并找到了一种方法。
Try this:
尝试这个：
seq 10 | xargs -i --max-procs=4 bash -c "echo start {}; sleep 3; echo done {}"
--max-procs=4will ensure that no more than four subprocesses are running at a time.
--max-procs=4将确保一次运行的子进程不超过四个。
The output will look like this:
输出将如下所示：
start 2
start 3
start 1
start 4
done 2
done 3
done 1
done 4
start 6
start 5
start 7
start 8
done 6
done 5
start 9
done 8
done 7
start 10
done 9
done 10
Note that the order of execution might not follow the commands in the order you submit them. As you can see 2 started before 1.
请注意，执行顺序可能不遵循您提交命令的顺序。如您所见，2 在 1 之前开始。
回答by mob
Quick and dirty solution: insert this line somewhere inside your forloop:
快速而肮脏的解决方案：在for循环中的某处插入此行：
while [ $(jobs | wc -l) -ge 4 ] ; do sleep 1 ; done
(assumes you don't already have other background jobs running in the same shell)
（假设您还没有在同一个 shell 中运行其他后台作业）
回答by Lynch
I have found another solution for this question using parallel(part of moreutilspackage.)
我使用parallel（moreutils包的一部分）为这个问题找到了另一个解决方案。
parallel -j 4 -i bash -c "echo start {}; sleep 2; echo done {};" -- $(seq 10)
-j 4stands for -j maxjobs
-j 4代表 -j maxjobs
-iuses the parameters as {}
-i使用参数作为 {}
--delimits your arguments
--界定你的论点
The output of this command will be:
此命令的输出将是：
start 3
start 4
start 1
start 2
done 4
done 2
done 3
done 1
start 5
start 6
start 7
start 8
done 5
done 6
start 9
done 7
start 10
done 8
done 9
done 10
回答by Mat
You can do something like this by using the jobsbuiltin:
你可以使用jobs内置函数来做这样的事情：
for f in $TESTS; do
running=($(jobs -rp))
while [ ${#running[@]} -ge 4 ] ; do
sleep 1 # this is not optimal, but you can't use wait here
running=($(jobs -rp))
done
t=$[ ( $RANDOM % 5 ) + 1 ]
sleep $t && echo $f $t &
done
wait
回答by Ole Tange
GNU Parallel is designed for this kind of tasks:
GNU Parallel 专为此类任务而设计：
TESTS="a b c d e"
for f in $TESTS; do
t=$[ ( $RANDOM % 5 ) + 1 ]
sem -j4 sleep $t && echo $f $t
done
sem --wait
Watch the intro videos to learn more:
观看介绍视频以了解更多信息：
回答by Seth Robertson
This tested script runs 5 jobs at a time and will restart a new job as soon as it does (due to the kill of the sleep 10.9 when we get a SIGCHLD. A simpler version of this could use direct polling (change the sleep 10.9 to sleep 1 and get rid of the trap).
这个经过测试的脚本一次运行 5 个作业，并且会尽快重新启动一个新作业（由于在我们获得 SIGCHLD 时会终止 sleep 10.9。这个更简单的版本可以使用直接轮询（将 sleep 10.9 更改为sleep 1 并摆脱陷阱）。
#!/usr/bin/bash
set -o monitor
trap "pkill -P $$ -f 'sleep 10\.9' >&/dev/null" SIGCHLD
totaljobs=15
numjobs=5
worktime=10
curjobs=0
declare -A pidlist
dojob()
{
slot=
time=$(echo "$RANDOM * 10 / 32768" | bc -l)
echo Starting job $slot with args $time
sleep $time &
pidlist[$slot]=`jobs -p %%`
curjobs=$(($curjobs + 1))
totaljobs=$(($totaljobs - 1))
}
# start
while [ $curjobs -lt $numjobs -a $totaljobs -gt 0 ]
do
dojob $curjobs
done
# Poll for jobs to die, restarting while we have them
while [ $totaljobs -gt 0 ]
do
for ((i=0;$i < $curjobs;i++))
do
if ! kill -0 ${pidlist[$i]} >&/dev/null
then
dojob $i
break
fi
done
sleep 10.9 >&/dev/null
done
wait
