Python 线程都在单个内核上执行
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/4496680/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Python threads all executing on a single core
提问by Rob Lourens
I have a Python program that spawns many threads, runs 4 at a time, and each performs an expensive operation. Pseudocode:
我有一个 Python 程序,它产生许多线程,一次运行 4 个,每个执行一个昂贵的操作。伪代码:
for object in list:
t = Thread(target=process, args=(object))
# if fewer than 4 threads are currently running, t.start(). Otherwise, add t to queue
But when the program is run, Activity Monitor in OS X shows that 1 of the 4 logical cores is at 100% and the others are at nearly 0. Obviously I can't force the OS to do anything but I've never had to pay attention to performance in multi-threaded code like this before so I was wondering if I'm just missing or misunderstanding something.
但是当程序运行时,OS X 中的 Activity Monitor 显示 4 个逻辑核心中有 1 个处于 100%,其他接近 0。显然我不能强迫操作系统做任何事情,但我从来没有必要这样做之前注意多线程代码中的性能,所以我想知道我是否只是遗漏或误解了某些东西。
Thanks.
谢谢。
采纳答案by Gabriel Grant
Note that in many cases (and virtually all cases where your "expensive operation" is a calculation implemented in Python), multiple threads will not actually run concurrently due to Python's Global Interpreter Lock (GIL).
请注意,在许多情况下(以及几乎所有“昂贵的操作”都是在 Python 中实现的计算),由于 Python 的全局解释器锁 (GIL),多个线程实际上不会并发运行。
The GIL is an interpreter-level lock. This lock prevents execution of multiple threads at once in the Python interpreter. Each thread that wants to run must wait for the GIL to be released by the other thread, which means your multi-threaded Python application is essentially single threaded, right? Yes. Not exactly. Sort of.
CPython uses what's called “operating system” threads under the covers, which is to say each time a request to make a new thread is made, the interpreter actually calls into the operating system's libraries and kernel to generate a new thread. This is the same as Java, for example. So in memory you really do have multiple threads and normally the operating system controls which thread is scheduled to run. On a multiple processor machine, this means you could have many threads spread across multiple processors, all happily chugging away doing work.
However, while CPython does use operating system threads (in theory allowing multiple threads to execute within the interpreter simultaneously), the interpreter also forces the GIL to be acquired by a thread before it can access the interpreter and stack and can modify Python objects in memory all willy-nilly. The latter point is why the GIL exists: The GIL prevents simultaneous access to Python objects by multiple threads. But this does not save you (as illustrated by the Bank example) from being a lock-sensitive creature; you don't get a free ride. The GIL is there to protect the interpreters memory, not your sanity.
GIL 是解释器级别的锁。此锁可防止在 Python 解释器中一次执行多个线程。每个想要运行的线程都必须等待另一个线程释放 GIL,这意味着您的多线程 Python 应用程序本质上是单线程的,对吗?是的。不完全是。有点。
CPython 在幕后使用了所谓的“操作系统”线程,也就是说,每次请求创建一个新线程时,解释器实际上都会调用操作系统的库和内核来生成一个新线程。例如,这与 Java 相同。所以在内存中你确实有多个线程,通常操作系统控制计划运行哪个线程。在多处理器机器上,这意味着您可以将多个线程分布在多个处理器上,所有线程都可以愉快地工作。
然而,虽然 CPython 确实使用操作系统线程(理论上允许多个线程在解释器内同时执行),解释器还强制线程在访问解释器和堆栈之前获取 GIL,并可以修改内存中的 Python 对象一切都愿意。后一点是 GIL 存在的原因:GIL 防止多个线程同时访问 Python 对象。但这并不能使您(如银行示例所示)免于成为对锁定敏感的生物;你不能搭便车。GIL 是为了保护口译员的记忆,而不是你的理智。
See the Global Interpreter Lock section of Jesse Noller's postfor more details.
有关更多详细信息,请参阅Jesse Noller 帖子的 Global Interpreter Lock 部分。
To get around this problem, check out Python's multiprocessing module.
要解决这个问题,请查看Python 的多处理模块。
multiple processes (with judicious use of IPC) are[...] a much better approach to writing apps for multi-CPU boxes than threads.
多进程(明智地使用 IPC)是[...] 比线程更好的为多 CPU 盒编写应用程序的方法。
回答by T.R.
Python has a Global Interpreter Lock, which can prevent threads of interpreted code from being processed concurrently.
Python 有一个全局解释器锁,可以防止解释代码的线程被并发处理。
http://en.wikipedia.org/wiki/Global_Interpreter_Lock
http://en.wikipedia.org/wiki/Global_Interpreter_Lock
http://wiki.python.org/moin/GlobalInterpreterLock
http://wiki.python.org/moin/GlobalInterpreterLock
For ways to get around this, try the multiprocessing module, as advised here:
有关解决此问题的方法,请按照此处的建议尝试使用multiprocessing 模块:
回答by MAK
AFAIK, in CPython the Global Interpreter Lock means that there can't be more than one block of Python code being run at any one time. Although this does not really affect anything in a single processor/single-core machine, on a mulitcore machine it means you have effectively only one thread running at any one time - causing all the other core to be idle.
AFAIK,在 CPython 中,全局解释器锁意味着在任何时候都不能运行超过一个 Python 代码块。尽管这不会真正影响单处理器/单核机器中的任何内容,但在多核机器上,这意味着您在任何时候实际上只有一个线程在运行 - 导致所有其他内核空闲。

