如何同时运行两个 python 循环?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/3474382/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-18 11:15:19  来源:igfitidea点击:

How do I run two python loops concurrently?

pythonconcurrency

提问by hiiii

Suppose I have the following in Python

假设我在 Python 中有以下内容

# A loop
for i in range(10000):
    Do Task A

# B loop
for i in range(10000):
    Do Task B

How do I run these loops simultaneously in Python?

如何在 Python 中同时运行这些循环?

回答by Dave Kirby

Why do you want to run the two processes at the same time? Is it because you think they will go faster (there is a good chance that they wont). Why not run the tasks in the same loop, e.g.

为什么要同时运行这两个进程?是不是因为你认为他们会走得更快(他们很可能不会)。为什么不在同一个循环中运行任务,例如

for i in range(10000):
    doTaskA()
    doTaskB()

The obvious answer to your question is to use threads - see the python threadingmodule. However threading is a big subject and has many pitfalls, so read up on it before you go down that route.

您的问题的明显答案是使用线程 - 请参阅 python线程模块。然而线程是一个很大的主题并且有很多陷阱,所以在你走那条路之前先阅读它。

Alternatively you could run the tasks in separate proccesses, using the python multiprocessingmodule. If both tasks are CPU intensive this will make better use of multiple cores on your computer.

或者,您可以使用 python多处理模块在单独的进程中运行任务。如果这两个任务都是 CPU 密集型的,这将更好地利用计算机上的多个内核。

There are other options such as coroutines, stackless tasklets, greenlets, CSP etc, but Without knowing more about Task A and Task B and why they need to be run at the same time it is impossible to give a more specific answer.

还有其他选项,例如协程、无堆栈小任务、greenlets、CSP 等,但是如果不了解更多关于任务 A 和任务 B 以及为什么需要同时运行它们,则无法给出更具体的答案。

回答by PeterK

How about: A loop for i in range(10000): Do Task A, Do Task B ? Without more information i dont have a better answer.

怎么样:范围内的 i 循环(10000):执行任务 A,执行任务 B ?没有更多信息,我没有更好的答案。

回答by Matt Curtis

You could use threadingor multiprocessing.

您可以使用threadingmultiprocessing

回答by Odomontois

from threading import Thread
def loopA():
    for i in range(10000):
        #Do task A
def loopB():
    for i in range(10000):
        #Do task B
threadA = Thread(target = loopA)
threadB = Thread(target = loobB)
threadA.run()
threadB.run()
# Do work indepedent of loopA and loopB 
threadA.join()
threadB.join()

回答by Stefano Palazzo

If you want concurrency, here's a very simple example:

如果你想要并发,这是一个非常简单的例子:

from multiprocessing import Process

def loop_a():
    while 1:
        print("a")

def loop_b():
    while 1:
        print("b")

if __name__ == '__main__':
    Process(target=loop_a).start()
    Process(target=loop_b).start()

This is just the most basic exampleI could think of. Be sure to read http://docs.python.org/library/multiprocessing.htmlto understand what's happening.

这只是我能想到的最基本的例子。请务必阅读http://docs.python.org/library/multiprocessing.html以了解发生了什么。

If you want to send data back to the program, I'd recommend using a Queue (which in my experience is easiest to use).

如果您想将数据发送回程序,我建议使用队列(根据我的经验,这是最容易使用的)。

You can use a thread instead if you don't mind the global interpreter lock. Processes are more expensive to instantiate but they offer true concurrency.

如果您不介意全局解释器 lock ,则可以改用线程。进程实例化成本更高,但它们提供真正的并发性。

回答by cizixs

There are many possible options for what you wanted:

您想要的有很多可能的选择:

use loop

使用循环

As many people have pointed out, this is the simplest way.

正如许多人指出的那样,这是最简单的方法。

for i in xrange(10000):
    # use xrange instead of range
    taskA()
    taskB()

Merits: easy to understand and use, no extra library needed.

优点:易于理解和使用,不需要额外的库。

Drawbacks: taskB must be done after taskA, or otherwise. They can't be running simultaneously.

缺点:taskB 必须在 taskA 之后完成,否则。它们不能同时运行。

multiprocess

多进程

Another thought would be: run two processes at the same time, python provides multiprocess library, the following is a simple example:

另一种想法是:同时运行两个进程,python提供了进程,下面是一个简单的例子:

from multiprocessing import Process


p1 = Process(target=taskA, args=(*args, **kwargs))
p2 = Process(target=taskB, args=(*args, **kwargs))

p1.start()
p2.start()

merits: task can be run simultaneouslyin the background, you can control tasks(end, stop them etc), tasks can exchange data, can be synchronized if they compete the same resources etc.

优点:任务可以simultaneously在后台运行,您可以控制任务(结束,停止它们等),任务可以交换数据,如果它们竞争相同的资源可以同步等。

drawbacks: too heavy!OS will frequently switch between them, they have their own data space even if data is redundant. If you have a lot tasks (say 100 or more), it's not what you want.

缺点:太重了!OS会在它们之间频繁切换,即使数据是冗余的,它们也有自己的数据空间。如果你有很多任务(比如 100 个或更多),这不是你想要的。

threading

穿线

threading is like process, just lightweight. check out this post. Their usage is quite similar:

线程就像进程,只是轻量级的。看看这篇文章。它们的用法非常相似:

import threading 


p1 = threading.Thread(target=taskA, args=(*args, **kwargs))
p2 = threading.Thread(target=taskB, args=(*args, **kwargs))

p1.start()
p2.start()

coroutines

协程

libraries like greenletand geventprovides something called coroutines, which is supposed to be faster than threading. No examples provided, please google how to use them if you're interested.

库喜欢greenletgevent提供了一种叫做协程的东西,它应该比线程更快。没有提供示例,如果您有兴趣,请谷歌如何使用它们。

merits: more flexible and lightweight

优点:更加灵活轻便

drawbacks: extra library needed, learning curve.

缺点:需要额外的库,学习曲线。

回答by Patrick Kantorski

I find that using the "pool" submodule within "multiprocessing" works amazingly for executing multiple processes at once within a Python Script.

我发现在“multiprocessing”中使用“pool”子模块非常适合在 Python 脚本中同时执行多个进程。

See Section: Using a pool of workers

请参阅部分:使用工人池

Look carefully at "# launching multiple evaluations asynchronously mayuse more processes" in the example. Once you understand what those lines are doing, the following example I constructed will make a lot of sense.

仔细看例子中的“#launch multiple evaluations asynchronously mayuse more processes”。一旦您了解了这些行的作用,我构建的以下示例就会很有意义。

import numpy as np
from multiprocessing import Pool

def desired_function(option, processes, data, etc...):
    # your code will go here. option allows you to make choices within your script
    # to execute desired sections of code for each pool or subprocess.

    return result_array   # "for example"


result_array = np.zeros("some shape")  # This is normally populated by 1 loop, lets try 4.
processes = 4
pool = Pool(processes=processes)
args = (processes, data, etc...)    # Arguments to be passed into desired function.

multiple_results = []
for i in range(processes):          # Executes each pool w/ option (1-4 in this case).
    multiple_results.append(pool.apply_async(param_process, (i+1,)+args)) # Syncs each.

results = np.array(res.get() for res in multiple_results)  # Retrieves results after
                                                           # every pool is finished!

for i in range(processes):
    result_array = result_array + results[i]  # Combines all datasets!

The code will basically run the desired function for a set number of processes. You will have to carefully make sure your function can distinguish between each process (hence why I added the variable "option".) Additionally, it doesn't have to be an array that is being populated in the end, but for my example, that's how I used it. Hope this simplifies or helps you better understand the power of multiprocessing in Python!

代码基本上会为一定数量的进程运行所需的功能。您必须仔细确保您的函数可以区分每个进程(因此我添加了变量“选项”。)此外,它不必是最后填充的数组,但对于我的示例,我就是这样用的。希望这可以简化或帮助您更好地理解 Python 中多处理的强大功能!