python 队列和多处理队列:它们的行为如何?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/925100/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-11-03 21:07:58  来源:igfitidea点击:

python queue & multiprocessing queue: how they behave?

pythonqueue

提问by DrFalk3n

This sample code works (I can write something in the file):

此示例代码有效(我可以在文件中写入一些内容):

from multiprocessing import Process, Queue

queue = Queue()
def _printer(self, queue):
    queue.put("hello world!!")

def _cmdDisp(self, queue):
    f = file("Cmd.log", "w")
    print >> f, queue.get()
    f.close()

instead this other sample not: (errormsg: 'module' object is not callable)

而不是这个其他示例:(错误消息:“模块”对象不可调用)

import Queue

queue = Queue()
def _printer(self, queue):
    queue.put("hello world!!")

def _cmdDisp(self, queue):
    f = file("Cmd.log", "w")
    print >> f, queue.get()
    f.close()

this other sample not (I cannot write something in the file):

这个其他样本不是(我不能在文件中写一些东西):

import Queue

queue = Queue.Queue()
def _printer(self, queue):
    queue.put("hello world!!")

def _cmdDisp(self, queue):
    f = file("Cmd.log", "w")
    print >> f, queue.get()
    f.close()

Can someone explain the differences? and the right to do?

有人可以解释这些差异吗?和权利做什么?

回答by Torsten Marek

For your second example, you already gave the explanation yourself---Queueis a module, which cannot be called.

对于你的第二个例子,你自己已经给出了解释---Queue是一个不能被调用的模块。

For the third example: I assume that you use Queue.Queuetogether with multiprocessing. A Queue.Queuewill not be shared between processes. If the Queue.Queueis declared before the processes then each process will receive a copy of it which is then independent of every other process. Items placed in the Queue.Queueby the parent before starting the children will be available to each child. Items placed in the Queue.Queueby the parent after starting the child will only be available to the parent. Queue.Queueis made for data interchange between different threadsinside the same process (using the threadingmodule). The multiprocessing queues are for data interchange between different Python processes. While the API looks similar (it's designed to be that way), the underlying mechanisms are fundamentally different.

对于第三个示例:我假设您Queue.Queuemultiprocessing. AQueue.Queue不会在进程之间共享。如果在Queue.Queue进程之前声明了 ,那么每个进程都会收到它的副本,然后独立于其他进程。Queue.Queue每个孩子都可以使用在开始孩子之前由父母放置的物品。Queue.Queue启动子项后由父项放置在 中的项目仅对父项可用。Queue.Queue用于同一进程内不同线程之间的数据交换(使用线程模块)。多处理队列用于不同 Python进程之间的数据交换. 虽然 API 看起来很相似(它就是这样设计的),但底层机制却有着根本的不同。

  • multiprocessingqueues exchange data by pickling (serializing) objects and sending them through pipes.
  • Queue.Queueuses a data structure that is shared between threads and locks/mutexes for correct behaviour.
  • multiprocessing队列通过酸洗(序列化)对象并通过管道发送它们来交换数据。
  • Queue.Queue使用在线程和锁/互斥锁之间共享的数据结构以获得正确的行为。