bash while 循环线程

bash while loop threading

提问by radman

i have a while loop reading lines from a $hosts

我有一个while循环从a读取行 $hosts

while read line
do
    ip=$line
    check
done < $hosts

my question is can I use some way to speed this up or run the check on 10 hosts at a time and each check is on a different IP and finish when all IP in $hosthave been checked? Thanks

我的问题是我可以使用某种方法来加快速度或一次在 10 台主机上运行检查，每次检查都在不同的 IP 上，并在$host检查完所有 IP 后完成吗？谢谢

回答by fejese

You can send tasks to the background by &If you intend to wait for all of them to finish you can use the waitcommand:

您可以通过以下方式将任务发送到后台&如果您打算等待所有任务完成，您可以使用以下wait命令：

process_to_background &
echo Processing ...
wait
echo Done

You can get the pidof the given task started in the background if you want to wait for one (or few) specific tasks.

pid如果您想等待一个（或几个）特定任务，您可以在后台启动给定任务。

important_process_to_background &
important_pid=$!
while i in {1..10}; do
    less_important_process_to_background $i &
done

wait $important_pid
echo Important task finished

wait
echo All tasks finished

On note though: the background processes can mess up the output as they will run asynchronously. You might want to use a named pipe to collect the output from them.

但请注意：后台进程可能会弄乱输出，因为它们将异步运行。您可能希望使用命名管道来收集它们的输出。

edit

编辑

As asked in the comments there might be a need for limiting the background processes forked. In this case you can keep track of how many background processes you've started and communicate with them through a named pipe.

正如评论中所问，可能需要限制分叉的后台进程。在这种情况下，您可以跟踪已启动的后台进程的数量，并通过命名管道与它们通信。

mkfifo tmp # creating named pipe

counter=0
while read ip
do
  if [ $counter -lt 10 ]; then # we are under the limit
    { check $ip; echo 'done' > tmp; } &
    let $[counter++];
  else
    read x < tmp # waiting for a process to finish
    { check $ip; echo 'done' > tmp; } &
  fi
done
cat /tmp > /dev/null # let all the background processes end

rm tmp # remove fifo

回答by P.P

You can start multiple processes, each calling the function checkand wait for them to finish.

您可以启动多个进程，每个进程调用该函数check并等待它们完成。

while read line 
do 
  ip=$line
  check &
done < $hosts
wait # wait for all child processes to finish

Whether this increases the speeddepends on available processors and the function check's implementation. You have to ensure there's no data dependency in checkbetween iterations.

这是否会提高速度取决于可用的处理器和函数check的实现。您必须确保check迭代之间没有数据依赖性。

回答by Geremia

Use GNU Parallel:

使用GNU 并行：

parallel check ::: $hosts