Windows上的Python-如何等待多个子进程?
时间:2020-03-06 14:25:13 来源:igfitidea点击:
如何在Windows上的Python中等待多个子进程,而无需主动等待(轮询)?这样的事情几乎对我有用:
proc1 = subprocess.Popen(['python','mytest.py']) proc2 = subprocess.Popen(['python','mytest.py']) proc1.wait() print "1 finished" proc2.wait() print "2 finished"
问题是当proc2在proc1之前完成时,父进程仍将等待proc1. 在Unix上,人们会在循环中使用" waitpid(0)"来获取子进程的返回码,因为它们完成了如何在Windows上的Python中实现类似的目的?
解决方案
可能看起来有些矫kill过正,但是,在这里:
import Queue, thread, subprocess results= Queue.Queue() def process_waiter(popen, description, que): try: popen.wait() finally: que.put( (description, popen.returncode) ) process_count= 0 proc1= subprocess.Popen( ['python', 'mytest.py'] ) thread.start_new_thread(process_waiter, (proc1, "1 finished", results)) process_count+= 1 proc2= subprocess.Popen( ['python', 'mytest.py'] ) thread.start_new_thread(process_waiter, (proc2, "2 finished", results)) process_count+= 1 # etc while process_count > 0: description, rc= results.get() print "job", description, "ended with rc =", rc process_count-= 1
Twisted具有可在Windows上运行的异步进程生成API。实际上,有几种不同的实现,其中很多都不是很好,但是我们可以在不更改代码的情况下在它们之间进行切换。
Windows上的Twisted将在后台自动执行等待。如果不想使用线程,则必须使用win32 API以避免轮询。像这样的东西:
import win32process import win32event # Note: CreateProcess() args are somewhat cryptic, look them up on MSDN proc1, thread1, pid1, tid1 = win32process.CreateProcess(...) proc2, thread2, pid2, tid2 = win32process.CreateProcess(...) thread1.close() thread2.close() processes = {proc1: "proc1", proc2: "proc2"} while processes: handles = processes.keys() # Note: WaitForMultipleObjects() supports at most 64 processes at a time index = win32event.WaitForMultipleObjects(handles, False, win32event.INFINITE) finished = handles[index] exitcode = win32process.GetExitCodeProcess(finished) procname = processes.pop(finished) finished.close() print "Subprocess %s finished with exit code %d" % (procname, exitcode)
在zseil的答案的基础上,我们可以结合使用子进程和win32 API调用来完成此任务。我使用了直接ctypes,因为我的Python刚好没有安装win32api。我仅以MSYS为例生成sleep.exe,但显然我们可以生成所需的任何进程。我使用OpenProcess()从进程的PID获取HANDLE,然后使用WaitForMultipleObjects等待任何进程完成。
import ctypes, subprocess from random import randint SYNCHRONIZE=0x00100000 INFINITE = -1 numprocs = 5 handles = {} for i in xrange(numprocs): sleeptime = randint(5,10) p = subprocess.Popen([r"c:\msys.0\bin\sleep.exe", str(sleeptime)], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False) h = ctypes.windll.kernel32.OpenProcess(SYNCHRONIZE, False, p.pid) handles[h] = p.pid print "Spawned Process %d" % p.pid while len(handles) > 0: print "Waiting for %d children..." % len(handles) arrtype = ctypes.c_long * len(handles) handle_array = arrtype(*handles.keys()) ret = ctypes.windll.kernel32.WaitForMultipleObjects(len(handle_array), handle_array, False, INFINITE) h = handle_array[ret] ctypes.windll.kernel32.CloseHandle(h) print "Process %d done" % handles[h] del handles[h] print "All done!"