Windows 上的 Python - 如何等待多个子进程？

Python on Windows - how to wait for multiple child processes?

提问by Rafa? Dowgird

How to wait for multiple child processes in Python on Windows, without active wait (polling)? Something like this almostworks for me:

如何在 Windows 上的 Python 中等待多个子进程，而无需主动等待（轮询）？这样的事情几乎对我有用：

proc1 = subprocess.Popen(['python','mytest.py'])
proc2 = subprocess.Popen(['python','mytest.py'])    
proc1.wait()
print "1 finished"
proc2.wait()
print "2 finished"

The problem is that when proc2finishes before proc1, the parent process will still wait for proc1. On Unix one would use waitpid(0)in a loop to get the child processes' return codes as they finish - how to achieve something like this in Python on Windows?

问题是在proc2之前完成时proc1，父进程仍然会等待proc1. 在 Unix 上，人们会waitpid(0)在循环中使用以在子进程完成时获取它们的返回码——如何在 Windows 上的 Python 中实现这样的目标？

采纳答案by tzot

It might seem overkill, but, here it goes:

这似乎有点矫枉过正，但是，这里是：

import Queue, thread, subprocess

results= Queue.Queue()
def process_waiter(popen, description, que):
    try: popen.wait()
    finally: que.put( (description, popen.returncode) )
process_count= 0

proc1= subprocess.Popen( ['python', 'mytest.py'] )
thread.start_new_thread(process_waiter,
    (proc1, "1 finished", results))
process_count+= 1

proc2= subprocess.Popen( ['python', 'mytest.py'] )
thread.start_new_thread(process_waiter,
    (proc2, "2 finished", results))
process_count+= 1

# etc

while process_count > 0:
    description, rc= results.get()
    print "job", description, "ended with rc =", rc
    process_count-= 1

回答by Glyph

Twisted has an asynchronous process-spawning APIwhich works on Windows. There are actually several different implementations, many of which are not so great, but you can switch between them without changing your code.

Twisted 有一个适用于 Windows的异步进程生成 API。实际上有几种不同的实现，其中许多都不是很好，但是您可以在不更改代码的情况下在它们之间切换。

回答by Ted Mielczarek

Building on zseil's answer, you can do this with a mix of subprocess and win32 API calls. I used straight ctypes, because my Python doesn't happen to have win32api installed. I'm just spawning sleep.exe from MSYS here as an example, but clearly you could spawn any process you like. I use OpenProcess() to get a HANDLE from the process' PID, and then WaitForMultipleObjects to wait for any process to finish.

基于 zseil 的回答，您可以混合使用 subprocess 和 win32 API 调用。我使用了直接的 ctypes，因为我的 Python 没有安装 win32api。我只是在这里从 MSYS 生成 sleep.exe 作为示例，但显然您可以生成任何您喜欢的进程。我使用 OpenProcess() 从进程的 PID 获取 HANDLE，然后使用 WaitForMultipleObjects 等待任何进程完成。

import ctypes, subprocess
from random import randint
SYNCHRONIZE=0x00100000
INFINITE = -1
numprocs = 5
handles = {}

for i in xrange(numprocs):
    sleeptime = randint(5,10)
    p = subprocess.Popen([r"c:\msys.0\bin\sleep.exe", str(sleeptime)], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
    h = ctypes.windll.kernel32.OpenProcess(SYNCHRONIZE, False, p.pid)
    handles[h] = p.pid
    print "Spawned Process %d" % p.pid

while len(handles) > 0:
    print "Waiting for %d children..." % len(handles)
    arrtype = ctypes.c_long * len(handles)
    handle_array = arrtype(*handles.keys())
    ret = ctypes.windll.kernel32.WaitForMultipleObjects(len(handle_array), handle_array, False, INFINITE)
    h = handle_array[ret]
    ctypes.windll.kernel32.CloseHandle(h)
    print "Process %d done" % handles[h]
    del handles[h]
print "All done!"

回答by user23475

Twisted on Windows will perform an active wait under the covers. If you don't want to use threads, you will have to use the win32 API to avoid polling. Something like this:

Windows 上的 Twisted 将在幕后执行主动等待。如果不想使用线程，则必须使用 win32 API 来避免轮询。像这样的东西：

import win32process
import win32event

# Note: CreateProcess() args are somewhat cryptic, look them up on MSDN
proc1, thread1, pid1, tid1 = win32process.CreateProcess(...)
proc2, thread2, pid2, tid2 = win32process.CreateProcess(...)
thread1.close()
thread2.close()

processes = {proc1: "proc1", proc2: "proc2"}

while processes:
    handles = processes.keys()
    # Note: WaitForMultipleObjects() supports at most 64 processes at a time
    index = win32event.WaitForMultipleObjects(handles, False, win32event.INFINITE)
    finished = handles[index]
    exitcode = win32process.GetExitCodeProcess(finished)
    procname = processes.pop(finished)
    finished.close()
    print "Subprocess %s finished with exit code %d" % (procname, exitcode)

回答by Giampaolo Rodolà

You can use psutil:

您可以使用psutil：

>>> import subprocess
>>> import psutil
>>> 
>>> proc1 = subprocess.Popen(['python','mytest.py'])
>>> proc2 = subprocess.Popen(['python','mytest.py'])    
>>> ls = [psutil.Process(proc1.pid), psutil.Process(proc2.pid)]
>>>
>>> gone, alive = psutil.wait_procs(ls, timeout=3)

'gone' and 'alive' are lists indicating which processes are gone and which ones are still alive.

'gone' 和 'alive' 是列表，表明哪些进程已经消失，哪些仍然活着。

Optionally you can specify a callback which gets invoked every time one of the watched processes terminates:

您可以选择指定一个回调，每次被监视的进程之一终止时都会调用该回调：

>>> def on_terminate(proc):
...     print "%s terminated" % proc
...
>>> gone, alive = psutil.wait_procs(ls, timeout=3, callback=on_terminate)