Python 多处理 for 循环？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/20190668/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Multiprocessing a for loop?
提问by ChrisFro
I have an array (called data_inputs) containing the names of hundreds of astronomy images files. These images are then manipulated. My code works and takes a few seconds to process each image. However, it can only do one image at a time because I'm running the array through a forloop:
我有一个数组（称为data_inputs），其中包含数百个天文图像文件的名称。然后处理这些图像。我的代码有效并且需要几秒钟来处理每个图像。但是，它一次只能处理一张图像，因为我通过for循环运行数组：
for name in data_inputs:
sci=fits.open(name+'.fits')
#image is manipulated
There is no reason why I have to modify an image before any other, so is it possible to utilise all 4 cores on my machine with each core running through the for loop on a different image?
没有理由我必须在任何其他图像之前修改图像，那么是否可以利用我机器上的所有 4 个核心，每个核心在不同的图像上通过 for 循环运行？
I've read about the multiprocessingmodule but I'm unsure how to implement it in my case.
I'm keen to get multiprocessingto work because eventually I'll have to run this on 10,000+ images.
我已经阅读了有关该multiprocessing模块的信息，但不确定如何在我的情况下实现它。我很想开始multiprocessing工作，因为最终我将不得不在 10,000 多张图像上运行它。
采纳答案by alko
You can simply use multiprocessing.Pool:
您可以简单地使用multiprocessing.Pool：
from multiprocessing import Pool
def process_image(name):
sci=fits.open('{}.fits'.format(name))
<process>
if __name__ == '__main__':
pool = Pool() # Create a multiprocessing Pool
pool.map(process_image, data_inputs) # process data_inputs iterable with pool
回答by ixxo
You can use multiprocessing.Pool:
您可以使用multiprocessing.Pool：
from multiprocessing import Pool
class Engine(object):
def __init__(self, parameters):
self.parameters = parameters
def __call__(self, filename):
sci = fits.open(filename + '.fits')
manipulated = manipulate_image(sci, self.parameters)
return manipulated
try:
pool = Pool(8) # on 8 processors
engine = Engine(my_parameters)
data_outputs = pool.map(engine, data_inputs)
finally: # To make sure processes are closed in the end, even if errors happen
pool.close()
pool.join()
回答by Spas
Alternatively
或者
with Pool() as pool:
pool.map(fits.open, [name + '.fits' for name in datainput])
