如何从 bash 脚本并行运行多个程序？

How do you run multiple programs in parallel from a bash script?

提问by Betamoo

I am trying to write a .sh filethat runs many programs simultaneously

我正在尝试编写一个同时运行多个程序的.sh 文件

I tried this

我试过这个

prog1 
prog2

But that runs prog1 then waits until prog1 ends and then starts prog2...

但是它运行 prog1 然后等到 prog1 结束然后启动 prog2 ...

So how can I run them in parallel?

那么我怎样才能并行运行它们呢？

采纳答案by psmears

prog1 &
prog2 &

回答by Ory Band

How about:

怎么样：

prog1 & prog2 && fg

This will:

这会：

1. Start prog1.
2. Send it to background, but keep printing its output.
3. Start prog2, and keep it in foreground, so you can close it with ctrl-c.
4. When you close prog2, you'll return to prog1's foreground, so you can also close it with ctrl-c.

1. 开始prog1。
2. 将其发送到后台，但继续打印其输出。
3. 开始prog2，并将其保留在前台，以便您可以使用ctrl-c.
4. 当你接近prog2，你会回到prog1的前景，所以你还可以关闭它ctrl-c。

回答by trusktr

You can use wait:

您可以使用wait：

some_command &
P1=$!
other_command &
P2=$!
wait $P1 $P2

It assigns the background program PIDs to variables ($!is the last launched process' PID), then the waitcommand waits for them. It is nice because if you kill the script, it kills the processes too!

它将后台程序 PID 分配给变量（$!是最后启动的进程的 PID），然后wait命令等待它们。这很好，因为如果您杀死脚本，它也会杀死进程！

回答by Ole Tange

With GNU Parallel http://www.gnu.org/software/parallel/it is as easy as:

使用 GNU Parallel http://www.gnu.org/software/parallel/就这么简单：

(echo prog1; echo prog2) | parallel

Or if you prefer:

或者，如果您更喜欢：

parallel ::: prog1 prog2

Learn more:

了解更多：

• Watch the intro video for a quick introduction: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
• Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.
• Read: Ole Tange, GNU Parallel 2018(Ole Tange, 2018).

• 观看介绍视频以进行快速介绍：https: //www.youtube.com/playlist?list=PL284C9FF2488BC6D1
• 完成教程 (man parallel_tutorial)。您的命令行会因此而爱您。
• 阅读：Ole Tange，GNU Parallel 2018（Ole Tange，2018）。

回答by Quinn Comendant

If you want to be able to easily run and kill multiple process with ctrl-c, this is my favorite method: spawn multiple background processes in a (…)subshell, and trap SIGINTto execute kill 0, which will kill everything spawned in the subshell group:

如果您希望能够轻松地使用 运行和杀死多个进程ctrl-c，这是我最喜欢的方法：在(…)子shell 中生成多个后台进程，并捕获SIGINT到 execute kill 0，这将杀死子shell 组中生成的所有内容：

(trap 'kill 0' SIGINT; prog1 & prog2 & prog3)

You can have complex process execution structures, and everything will close with a single ctrl-c(just make sure the last process is run in the foreground, i.e., don't include a &after prog1.3):

您可以拥有复杂的进程执行结构，并且所有内容都将以单个结束ctrl-c（只需确保最后一个进程在前台运行，即不包含&after prog1.3）：

(trap 'kill 0' SIGINT; prog1.1 && prog1.2 & (prog2.1 | prog2.2 || prog2.3) & prog1.3)

回答by mklement0

xargs -P <n>allows you to run <n>commands in parallel.

xargs -P <n>允许您<n>并行运行命令。

While -Pis a nonstandard option, both the GNU (Linux) and macOS/BSD implementations support it.

虽然-P是非标准选项，但 GNU (Linux) 和 macOS/BSD 实现都支持它。

The following example:

下面的例子：

• runs at most3 commands in parallel at a time,
• with additional commands starting only when a previously launched process terminates.

• 一次最多并行运行3 个命令，
• 附加命令仅在先前启动的进程终止时启动。

time xargs -P 3 -I {} sh -c 'eval ""' - {} <<'EOF'
sleep 1; echo 1
sleep 2; echo 2
sleep 3; echo 3
echo 4
EOF

The output looks something likes:

输出看起来像：

1   # output from 1st command 
4   # output from *last* command, which started as soon as the count dropped below 3
2   # output from 2nd command
3   # output from 3rd command

real    0m3.012s
user    0m0.011s
sys 0m0.008s

The timing shows that the commands were run in parallel (the last command was launched only after the first of the original 3 terminated, but executed very quickly).

计时显示命令是并行运行的（最后一个命令仅在原始 3 个命令中的第一个终止后才启动，但执行速度非常快）。

The xargscommand itself won't return until all commands have finished, but you can execute it in the background by terminating it with control operator &and then using the waitbuiltin to wait for the entire xargscommand to finish.

该xargs命令本身不会返回，直到所有的命令已经完成，但你可以在后台通过与控制操作终止执行它它&，然后使用wait内置等待整个xargs命令来完成。

{
  xargs -P 3 -I {} sh -c 'eval ""' - {} <<'EOF'
sleep 1; echo 1
sleep 2; echo 2
sleep 3; echo 3
echo 4
EOF
} &

# Script execution continues here while `xargs` is running 
# in the background.
echo "Waiting for commands to finish..."

# Wait for `xargs` to finish, via special variable $!, which contains
# the PID of the most recently started background process.
wait $!

Note:

笔记：

• BSD/macOS xargsrequires you to specify the count of commands to run in parallel explicitly, whereas GNU xargsallows you to specify -P 0to run as many as possiblein parallel.

• Output from the processes run in parallel arrives as it is being generated, so it will be unpredictably interleaved.

  • GNU parallel, as mentioned in Ole's answer(does notcome standard with most platforms), conveniently serializes(groups) the output on a per-process basis and offers many more advanced features.

• BSD / MacOS的xargs要求指定命令的计数并行运行明确，而GNUxargs允许你指定-P 0要尽可能多的运行尽可能平行。

• 并行运行的进程的输出在生成时到达，因此将不可预测地交错。

  • GNU parallel，如Ole 的回答中所提到的（大多数平台都没有标准），可以方便地在每个进程的基础上对输出进行序列化（分组），并提供许多更高级的功能。

回答by fermin

#!/bin/bash
prog1 & 2> .errorprog1.log; prog2 & 2> .errorprog2.log

Redirect errors to separate logs.

将错误重定向到单独的日志。

回答by 3h4x

There is a very useful program that calls nohup.

有一个非常有用的程序可以调用 nohup。

     nohup - run a command immune to hangups, with output to a non-tty

回答by arnaldocan

Here is a function I use in order to run at max n process in parallel (n=4 in the example):

这是我用来并行运行最大 n 个进程的函数（示例中 n=4）：

max_children=4

function parallel {
  local time1=$(date +"%H:%M:%S")
  local time2=""

  # for the sake of the example, I'm using  as a description, you may be interested in other description
  echo "starting  ($time1)..."
  "$@" && time2=$(date +"%H:%M:%S") && echo "finishing  ($time1 -- $time2)..." &

  local my_pid=$$
  local children=$(ps -eo ppid | grep -w $my_pid | wc -w)
  children=$((children-1))
  if [[ $children -ge $max_children ]]; then
    wait -n
  fi
}

parallel sleep 5
parallel sleep 6
parallel sleep 7
parallel sleep 8
parallel sleep 9
wait

If max_children is set to the number of cores, this function will try to avoid idle cores.

如果 max_children 设置为内核数，该函数将尽量避免空闲内核。

回答by Joaopcribeiro

I had a similar situation recently where I needed to run multiple programs at the same time, redirect their outputs to separated log files and wait for them to finish and I ended up with something like that:

我最近遇到了类似的情况，我需要同时运行多个程序，将它们的输出重定向到单独的日志文件并等待它们完成，结果是这样的：

#!/bin/bash

# Add the full path processes to run to the array
PROCESSES_TO_RUN=("/home/joao/Code/test/prog_1/prog1" \
                  "/home/joao/Code/test/prog_2/prog2")
# You can keep adding processes to the array...

for i in ${PROCESSES_TO_RUN[@]}; do
    ${i%/*}/./${i##*/} > ${i}.log 2>&1 &
    # ${i%/*} -> Get folder name until the /
    # ${i##*/} -> Get the filename after the /
done

# Wait for the processes to finish
wait

Source: http://joaoperibeiro.com/execute-multiple-programs-and-redirect-their-outputs-linux/

来源：http: //joaoperibeiro.com/execute-multiple-programs-and-redirect-their-outputs-linux/