windows shell进程的并行执行
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/672719/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Parallel execution of shell processes
提问by Dirk Vollmar
Is there a tool available to execute several process in parallel in a Windows batch file? I have found some interesting tools for Linux (paralleland PPSS), however, I would need a tool for Windows platforms.
是否有可用于在 Windows 批处理文件中并行执行多个进程的工具？我发现了一些有趣的 Linux 工具（并行和PPSS），但是，我需要一个适用于 Windows 平台的工具。
Bonus: It would be great if the tool also allowed to distribute processes in an easy way among several machines, running the processes remotely a la PsExec.
奖励：如果该工具还允许以简单的方式在多台机器之间分配进程，远程运行进程，那就太好了PsExec。
Example: I would like that in the following for loop
示例：我希望在以下 for 循环中
for %F in (*.*) do processFile.exe %F
a limited amount of instances of processFile.exe are running in parallel to take advantage of multi-core CPUs.
有限数量的 processFile.exe 实例并行运行以利用多核 CPU。
采纳答案by ADEpt
GNU xargs under Linux has a "-P n" switch to launch "n" processes in parallel.
Linux 下的 GNU xargs 有一个“-P n”开关来并行启动“n”个进程。
Maybe cygwin/mingw build of xargs also supports this?
也许 cygwin/mingw 构建的 xargs 也支持这个？
Then you can use:
然后你可以使用：
xargs -P 4 processFile < fileList
No fancy multi-node process spawning, though.
不过，没有花哨的多节点进程产生。
回答by dbenham
Edit- I modified the script to optionally display the output of each process
编辑-我修改了脚本以选择性地显示每个进程的输出
Here is a native batch solution that reliably runs a list of commands in parallel, never launching more than nprocesses at a time.
这是一个本地批处理解决方案，它可以可靠地并行运行一系列命令，一次从不启动超过n 个进程。
It even has a mechanism built in to distribute the processes to specific CPUs or remote machines via PSEXEC, but I haven't tested that feature.
它甚至内置了一种机制，可以通过 PSEXEC 将进程分发到特定的 CPU 或远程机器，但我还没有测试过该功能。
The trick to make this work is to START each command through a CMD process that redirects either stdout or an undefined handle to a lock file. The process will maintain an exclusive lock on the file until it terminates. It doesn't matter how the process terminates (normal exit, crash, killed process), the lock will be released as soon as it does.
完成这项工作的技巧是通过 CMD 进程启动每个命令，该进程将 stdout 或未定义的句柄重定向到锁定文件。该进程将保持对文件的排他锁，直到它终止。无论进程如何终止（正常退出、崩溃、杀死进程），锁都会尽快释放。
The master script can test if the process is still active by attempting to redirect to the same lock file. The redirection will fail if the process is still active, succeed if it has terminated.
主脚本可以通过尝试重定向到同一个锁定文件来测试进程是否仍然处于活动状态。如果进程仍然处于活动状态，重定向将失败，如果它已终止，则重定向成功。
By default, the script ignores the output of each process. If started with the /Ooption as the 1st parameter, then it displays the output of each process, without interleaving.
默认情况下，脚本会忽略每个进程的输出。如果以该/O选项作为第一个参数开始，则它会显示每个进程的输出，而不会交错。
My demo sets the process limit to 4, and simply runs a series of PING commands of varying length.
我的演示将进程限制设置为 4，并且只是运行一系列不同长度的 PING 命令。
I've tested this on XP, Vista, and Windows 7.
我已经在 XP、Vista 和 Windows 7 上对此进行了测试。
@echo off
setlocal enableDelayedExpansion
:: Display the output of each process if the /O option is used
:: else ignore the output of each process
if /i "%~1" equ "/O" (
set "lockHandle=1"
set "showOutput=1"
) else (
set "lockHandle=1^>nul 9"
set "showOutput="
)
:: The list of commands could come from anywhere such as another file
:: or the output of another command. For this demo I will list the
:: commands within this script - Each command is prefixed with :::
::: ping /n 05 ::1
::: ping /n 20 ::1
::: ping /n 10 ::1
::: ping /n 15 ::1
::: ping /n 07 ::1
::: ping /n 05 ::1
::: ping /n 20 ::1
::: ping /n 10 ::1
::: ping /n 15 ::1
::: ping /n 07 ::1
:: Define the maximum number of parallel processes to run.
:: Each process number can optionally be assigned to a particular server
:: and/or cpu via psexec specs (untested).
set "maxProc=4"
:: Optional - Define CPU targets in terms of PSEXEC specs
:: (everything but the command)
::
:: If a CPU is not defined for a proc, then it will be run on the local machine.
:: I haven't tested this feature, but it seems like it should work.
::
:: set cpu1=psexec \server1 ...
:: set cpu2=psexec \server1 ...
:: set cpu3=psexec \server2 ...
:: etc.
:: For this demo force all CPU specs to undefined (local machine)
for /l %%N in (1 1 %maxProc%) do set "cpu%%N="
:: Get a unique base lock name for this particular instantiation.
:: Incorporate a timestamp from WMIC if possible, but don't fail if
:: WMIC not available. Also incorporate a random number.
set "lock="
for /f "skip=1 delims=-+ " %%T in ('2^>nul wmic os get localdatetime') do (
set "lock=%%T"
goto :break
)
:break
set "lock=%temp%\lock%lock%_%random%_"
:: Initialize the counters
set /a "startCount=0, endCount=0"
:: Clear any existing end flags
for /l %%N in (1 1 %maxProc%) do set "endProc%%N="
:: Launch the commands in a loop
:: Modify the IN () clause as needed to retrieve the list of commands
set launch=1
for /f "tokens=* delims=:" %%A in ('findstr /b ":::" "%~f0"') do (
if !startCount! lss %maxProc% (
set /a "startCount+=1, nextProc=startCount"
) else (
call :wait
)
set cmd!nextProc!=%%A
if defined showOutput echo -------------------------------------------------------------------------------
echo !time! - proc!nextProc!: starting %%A
2>nul del %lock%!nextProc!
%= Redirect the lock handle to the lock file. The CMD process will =%
%= maintain an exclusive lock on the lock file until the process ends. =%
start /b "" cmd /c %lockHandle%^>"%lock%!nextProc!" 2^>^&1 !cpu%%N! %%A
)
set "launch="
:wait
:: Wait for procs to finish in a loop
:: If still launching then return as soon as a proc ends
:: else wait for all procs to finish
:: redirect stderr to null to suppress any error message if redirection
:: within the loop fails.
for /l %%N in (1 1 %startCount%) do 2>nul (
%= Redirect an unused file handle to the lock file. If the process is =%
%= still running then redirection will fail and the IF body will not run =%
if not defined endProc%%N if exist "%lock%%%N" 9>>"%lock%%%N" (
%= Made it inside the IF body so the process must have finished =%
if defined showOutput echo ===============================================================================
echo !time! - proc%%N: finished !cmd%%N!
if defined showOutput type "%lock%%%N"
if defined launch (
set nextProc=%%N
exit /b
)
set /a "endCount+=1, endProc%%N=1"
)
)
if %endCount% lss %startCount% (
1>nul 2>nul ping /n 2 ::1
goto :wait
)
2>nul del %lock%*
if defined showOutput echo ===============================================================================
echo Thats all folks^^!
Here is output from a sample run that ignores process output
这是忽略进程输出的示例运行的输出
12:24:07.52 - proc1: starting ping /n 05 ::1
12:24:07.52 - proc2: starting ping /n 20 ::1
12:24:07.53 - proc3: starting ping /n 10 ::1
12:24:07.54 - proc4: starting ping /n 15 ::1
12:24:11.60 - proc1: finished ping /n 05 ::1
12:24:11.60 - proc1: starting ping /n 07 ::1
12:24:16.66 - proc3: finished ping /n 10 ::1
12:24:16.66 - proc3: starting ping /n 05 ::1
12:24:17.68 - proc1: finished ping /n 07 ::1
12:24:17.68 - proc1: starting ping /n 20 ::1
12:24:20.72 - proc3: finished ping /n 05 ::1
12:24:20.72 - proc3: starting ping /n 10 ::1
12:24:21.75 - proc4: finished ping /n 15 ::1
12:24:21.75 - proc4: starting ping /n 15 ::1
12:24:26.82 - proc2: finished ping /n 20 ::1
12:24:26.82 - proc2: starting ping /n 07 ::1
12:24:29.86 - proc3: finished ping /n 10 ::1
12:24:32.89 - proc2: finished ping /n 07 ::1
12:24:35.92 - proc4: finished ping /n 15 ::1
12:24:36.93 - proc1: finished ping /n 20 ::1
Thats all folks!
Here is the output if run with the /Ooption showing process output
这是使用/O显示进程输出的选项运行时的输出
-------------------------------------------------------------------------------
12:24:51.02 - proc1: starting ping /n 05 ::1
-------------------------------------------------------------------------------
12:24:51.02 - proc2: starting ping /n 20 ::1
-------------------------------------------------------------------------------
12:24:51.03 - proc3: starting ping /n 10 ::1
-------------------------------------------------------------------------------
12:24:51.04 - proc4: starting ping /n 15 ::1
===============================================================================
12:24:55.10 - proc1: finished ping /n 05 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:24:55.10 - proc1: starting ping /n 07 ::1
===============================================================================
12:25:00.17 - proc3: finished ping /n 10 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 10, Received = 10, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:00.19 - proc3: starting ping /n 05 ::1
===============================================================================
12:25:01.22 - proc1: finished ping /n 07 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 7, Received = 7, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:01.23 - proc1: starting ping /n 20 ::1
===============================================================================
12:25:04.27 - proc3: finished ping /n 05 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:04.28 - proc3: starting ping /n 10 ::1
===============================================================================
12:25:05.30 - proc4: finished ping /n 15 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 15, Received = 15, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:05.32 - proc4: starting ping /n 15 ::1
===============================================================================
12:25:10.38 - proc2: finished ping /n 20 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:10.40 - proc2: starting ping /n 07 ::1
===============================================================================
12:25:13.44 - proc3: finished ping /n 10 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 10, Received = 10, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
12:25:16.48 - proc2: finished ping /n 07 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 7, Received = 7, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
12:25:19.52 - proc4: finished ping /n 15 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 15, Received = 15, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
12:25:20.54 - proc1: finished ping /n 20 ::1
Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Ping statistics for ::1:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
Thats all folks!
回答by guerda
Try start:
尝试start：
start "title of the process" "P:\ath\to.exe"
It opens a new window with the given title and executes the BAT, CMD or EXE file. You can also set the priority, set the same environment etc.
它会打开一个具有给定标题的新窗口并执行 BAT、CMD 或 EXE 文件。您还可以设置优先级，设置相同的环境等。
Files being not executeable are opened with the associated program.
不可执行的文件使用关联的程序打开。
Further reading: Start -> Run
进一步阅读：开始 -> 运行
cmd /k start /?
Start is available at least since WinME.
至少从 WinME 开始就可以使用 Start。
Good luck!
祝你好运！
回答by Joey
Sounds more like you want to use Powershell 2. However, you can spawn new cmdwindows (or other processes) by using start, see also thisanswer. Although you probably have to use some other tools and a little trickery to create something like a "process pool" (to have only a maximum of ninstances running at a time). You could achieve the latter by using tasklist /imand counting how many are already there (forloop or wc, if applicable) and simply wait (ping -n 2 ::1 >nul 2>&1) and re-check again whether you can spawn a new process.
听起来更像是您想使用 Powershell 2。但是，您可以使用 生成新cmd窗口（或其他进程）start，另请参阅此答案。尽管您可能需要使用一些其他工具和一些技巧来创建类似“进程池”的东西（一次最多只能运行n 个实例）。您可以通过使用tasklist /im和计算已经存在的数量（for循环或wc，如果适用）来实现后者，然后只需等待 ( ping -n 2 ::1 >nul 2>&1) 并再次检查是否可以生成新进程。
I have cobbled together a little test batch for this:
我为此拼凑了一个小测试批次：
@echo off
for /l %%i in (1,1,20) do call :loop %%i
goto :eof
:loop
call :checkinstances
if %INSTANCES% LSS 5 (
rem just a dummy program that waits instead of doing useful stuff
rem but suffices for now
echo Starting processing instance for %1
start /min wait.exe 5 sec
goto :eof
)
rem wait a second, can be adjusted with -w (-n 2 because the first ping returns immediately;
rem otherwise just use an address that's unused and -n 1)
echo Waiting for instances to close ...
ping -n 2 ::1 >nul 2>&1
rem jump back to see whether we can spawn a new process now
goto loop
goto :eof
:checkinstances
rem this could probably be done better. But INSTANCES should contain the number of running instances afterwards.
for /f "usebackq" %%t in (`tasklist /fo csv /fi "imagename eq wait.exe"^|find /c /v ""`) do set INSTANCES=%%t
goto :eof
It spawns a maximum of four new processes that execute in parallel and minimized. Wait time needs to be adjusted probably, depending on how much each process does and how long it is running. You probably also need to adjust the process name for which tasklist is looking if you're doing something else.
它最多产生四个并行执行并最小化的新进程。等待时间可能需要调整，这取决于每个进程做了多少以及运行了多长时间。如果您正在做其他事情，您可能还需要调整任务列表正在查找的进程名称。
There is no way to properly count the processes that are spawned by this batch, though. One way would be to create a random number at the start of the batch (%RANDOM%) and create a helper batch that does the processing (or spawns the processing program) but which can set its window title to a parameter:
但是，无法正确计算此批处理产生的进程。一种方法是在批处理开始时创建一个随机数 ( %RANDOM%) 并创建一个辅助批处理来执行处理（或生成处理程序），但可以将其窗口标题设置为参数：
@echo off
title %1
"%2" "%3"
This would be a simple batch that sets its title to the first parameter and then runs the second parameter with the third as argument. You can then filter in tasklist by selecting only processes with the specified window title (tasklist /fi "windowtitle eq ..."). This should work fairly reliable and prevents too many false positives. Searching for cmd.exewould be a bad idea if you still have some instances running, as that limits your pool of worker processes.
这将是一个简单的批处理，将其标题设置为第一个参数，然后使用第三个参数运行第二个参数。然后，您可以通过仅选择具有指定窗口标题 ( tasklist /fi "windowtitle eq ...") 的进程在任务列表中进行过滤。这应该工作相当可靠，并防止过多的误报。cmd.exe如果您仍有一些实例在运行，搜索将是一个坏主意，因为这会限制您的工作进程池。
You can use %NUMBER_OF_PROCESSORS%to create a sensible default of how many instances to spawn.
您可以使用它%NUMBER_OF_PROCESSORS%来创建一个合理的默认值来生成多少个实例。
You can also easily adapt this to use psexecto spawn the processes remotely (but wouldn't be very viable as you have to have admin privileges on the other machine as well as provide the password in the batch). You would have to use process names for filtering then, though.
您还可以轻松地调整它以用于psexec远程生成进程（但不太可行，因为您必须在另一台机器上拥有管理员权限并在批处理中提供密码）。但是，您必须使用进程名称进行过滤。
回答by PP.
There is a basic Windows xargs-like-clone which does support the -P parallel processing option at http://www.pirosa.co.uk/demo/wxargs/wxargs.html
在http://www.pirosa.co.uk/demo/wxargs/wxargs.html有一个基本的 Windows xargs-like-clone 支持 -P 并行处理选项
