如何在不冻结 PyQt GUI 的情况下跟踪 Python 中的线程进度？

How to keep track of thread progress in Python without freezing the PyQt GUI?

提问by tgray

Questions:

问题：

1. What is the best practice for keeping track of a tread's progress without locking the GUI ("Not Responding")?
2. Generally, what are the best practices for threading as it applies to GUI development?

1. 在不锁定 GUI（“无响应”）的情况下跟踪胎面进度的最佳做法是什么？
2. 一般来说，应用于 GUI 开发的线程的最佳实践是什么？

Question Background:

问题背景：

• I have a PyQt GUI for Windows.
• It is used to process sets of HTML documents.
• It takes anywhere from three seconds to three hours to process a set of documents.
• I want to be able to process multiple sets at the same time.
• I don't want the GUI to lock.
• I'm looking at the threading module to achieve this.
• I am relatively new to threading.
• The GUI has one progress bar.
• I want it to display the progress of the selected thread.
• Display results of the selected thread if it's finished.
• I'm using Python 2.5.

• 我有一个适用于 Windows 的 PyQt GUI。
• 它用于处理 HTML 文档集。
• 处理一组文档需要三秒钟到三个小时的时间。
• 我希望能够同时处理多组。
• 我不希望 GUI 锁定。
• 我正在查看线程模块来实现这一点。
• 我对线程比较陌生。
• GUI 有一个进度条。
• 我希望它显示所选线程的进度。
• 如果完成，则显示所选线程的结果。
• 我正在使用 Python 2.5。

My Idea:Have the threads emit a QtSignal when the progress is updated that triggers some function that updates the progress bar. Also signal when finished processing so results can be displayed.

我的想法：当进度更新时，让线程发出 QtSignal，触发一些更新进度条的函数。处理完成时也发出信号，以便显示结果。

#NOTE: this is example code for my idea, you do not have
#      to read this to answer the question(s).

import threading
from PyQt4 import QtCore, QtGui
import re
import copy

class ProcessingThread(threading.Thread, QtCore.QObject):

    __pyqtSignals__ = ( "progressUpdated(str)",
                        "resultsReady(str)")

    def __init__(self, docs):
        self.docs = docs
        self.progress = 0   #int between 0 and 100
        self.results = []
        threading.Thread.__init__(self)

    def getResults(self):
        return copy.deepcopy(self.results)

    def run(self):
        num_docs = len(self.docs) - 1
        for i, doc in enumerate(self.docs):
            processed_doc = self.processDoc(doc)
            self.results.append(processed_doc)
            new_progress = int((float(i)/num_docs)*100)

            #emit signal only if progress has changed
            if self.progress != new_progress:
                self.emit(QtCore.SIGNAL("progressUpdated(str)"), self.getName())
            self.progress = new_progress
            if self.progress == 100:
                self.emit(QtCore.SIGNAL("resultsReady(str)"), self.getName())

    def processDoc(self, doc):
        ''' this is tivial for shortness sake '''
        return re.findall('<a [^>]*>.*?</a>', doc)


class GuiApp(QtGui.QMainWindow):

    def __init__(self):
        self.processing_threads = {}  #{'thread_name': Thread(processing_thread)}
        self.progress_object = {}     #{'thread_name': int(thread_progress)}
        self.results_object = {}      #{'thread_name': []}
        self.selected_thread = ''     #'thread_name'

    def processDocs(self, docs):
        #create new thread
        p_thread = ProcessingThread(docs)
        thread_name = "example_thread_name"
        p_thread.setName(thread_name)
        p_thread.start()

        #add thread to dict of threads
        self.processing_threads[thread_name] = p_thread

        #init progress_object for this thread
        self.progress_object[thread_name] = p_thread.progress  

        #connect thread signals to GuiApp functions
        QtCore.QObject.connect(p_thread, QtCore.SIGNAL('progressUpdated(str)'), self.updateProgressObject(thread_name))
        QtCore.QObject.connect(p_thread, QtCore.SIGNAL('resultsReady(str)'), self.updateResultsObject(thread_name))

    def updateProgressObject(self, thread_name):
        #update progress_object for all threads
        self.progress_object[thread_name] = self.processing_threads[thread_name].progress

        #update progress bar for selected thread
        if self.selected_thread == thread_name:
            self.setProgressBar(self.progress_object[self.selected_thread])

    def updateResultsObject(self, thread_name):
        #update results_object for thread with results
        self.results_object[thread_name] = self.processing_threads[thread_name].getResults()

        #update results widget for selected thread
        try:
            self.setResultsWidget(self.results_object[thread_name])
        except KeyError:
            self.setResultsWidget(None)

Any commentary on this approach (e.g. drawbacks, pitfalls, praises, etc.) will be appreciated.

对这种方法的任何评论（例如缺点、陷阱、赞美等）将不胜感激。

Resolution:

解决：

I ended up using the QThread class and associated signals and slots to communicate between threads. This is primarily because my program already uses Qt/PyQt4 for the GUI objects/widgets. This solution also required fewer changes to my existing code to implement.

我最终使用 QThread 类和关联的信号和槽在线程之间进行通信。这主要是因为我的程序已经将 Qt/PyQt4 用于 GUI 对象/小部件。此解决方案还需要对我现有的代码进行较少的更改才能实现。

Here is a link to an applicable Qt article that explains how Qt handles threads and signals, http://www.linuxjournal.com/article/9602. Excerpt below:

这是一篇适用的 Qt 文章的链接，该文章解释了 Qt 如何处理线程和信号，http://www.linuxjournal.com/article/9602。摘录如下：

Fortunately, Qt permits signals and slots to be connected across threads—as long as the threads are running their own event loops. This is a much cleaner method of communication compared to sending and receiving events, because it avoids all the bookkeeping and intermediate QEvent-derived classes that become necessary in any nontrivial application. Communicating between threads now becomes a matter of connecting signals from one thread to the slots in another, and the mutexing and thread-safety issues of exchanging data between threads are handled by Qt.

Why is it necessary to run an event loop within each thread to which you want to connect signals? The reason has to do with the inter-thread communication mechanism used by Qt when connecting signals from one thread to the slot of another thread. When such a connection is made, it is referred to as a queued connection. When signals are emitted through a queued connection, the slot is invoked the next time the destination object's event loop is executed. If the slot had instead been invoked directly by a signal from another thread, that slot would execute in the same context as the calling thread. Normally, this is not what you want (and especially not what you want if you are using a database connection, as the database connection can be used only by the thread that created it). The queued connection properly dispatches the signal to the thread object and invokes its slot in its own context by piggy-backing on the event system. This is precisely what we want for inter-thread communication in which some of the threads are handling database connections. The Qt signal/slot mechanism is at root an implementation of the inter-thread event-passing scheme outlined above, but with a much cleaner and easier-to-use interface.

幸运的是，Qt 允许跨线程连接信号和槽——只要线程运行它们自己的事件循环。与发送和接收事件相比，这是一种更简洁的通信方法，因为它避免了在任何重要应用程序中变得必要的所有簿记和中间 QEvent 派生类。线程之间的通信现在变成了将信号从一个线程连接到另一个线程中的插槽的问题，并且线程之间交换数据的互斥和线程安全问题由 Qt 处理。

为什么需要在要连接信号的每个线程中运行事件循环？原因与 Qt 在将信号从一个线程连接到另一个线程的插槽时使用的线程间通信机制有关。当建立这样的连接时，它被称为排队连接。当通过排队连接发出信号时，下次执行目标对象的事件循环时会调用插槽。如果该槽被另一个线程的信号直接调用，则该槽将在与调用线程相同的上下文中执行。通常，这不是您想要的（尤其是在您使用数据库连接时不是您想要的，因为数据库连接只能由创建它的线程使用）。排队连接正确地将信号分派给线程对象，并通过事件系统捎带在其自己的上下文中调用其槽。这正是我们想要的线程间通信，其中一些线程正在处理数据库连接。Qt 信号/槽机制的根本是上面概述的线程间事件传递方案的实现，但具有更清晰和更易于使用的界面。

NOTE:elibenalso has a good answer, and if I weren't using PyQt4, which handles thread-safety and mutexing, his solution would have been my choice.

注意：eliben也有一个很好的答案，如果我不使用处理线程安全和互斥的 PyQt4，他的解决方案将是我的选择。

采纳答案by David Boddie

If you want to use signals to indicate progress to the main thread then you should really be using PyQt's QThread class instead of the Thread class from Python's threading module.

如果您想使用信号来指示主线程的进度，那么您真的应该使用 PyQt 的 QThread 类而不是 Python 线程模块中的 Thread 类。

A simple example which uses QThread, signals and slots can be found on the PyQt Wiki:

可以在 PyQt Wiki 上找到一个使用 QThread、信号和槽的简单示例：

https://wiki.python.org/moin/PyQt/Threading,_Signals_and_Slots

https://wiki.python.org/moin/PyQt/Threading,_Signals_and_Slots

回答by David Boddie

Native python queues won't work because you have to block on queue get(), which bungs up your UI.

本机 python 队列将无法工作，因为您必须阻塞队列 get()，这会阻塞您的 UI。

Qt essentially implements a queuing system on the inside for cross thread communication. Try this call from any thread to post a call to a slot.

Qt本质上在内部实现了一个队列系统，用于跨线程通信。尝试从任何线程调用此调用以发布对插槽的调用。

QtCore.QMetaObject.invokeMethod()

QtCore.QMetaObject.invokeMethod()

It's clunky and is poorly documented, but it should do what you want even from from a non-Qt thread.

它很笨重，文档也很差，但是即使从非 Qt 线程中，它也应该可以执行您想要的操作。

You can also use event machinery for this. See QApplication (or QCoreApplication) for a method named something like "post".

您也可以为此使用事件机制。有关名为“post”之类的方法，请参阅 QApplication（或 QCoreApplication）。

Edit: Here's a more complete example...

编辑：这是一个更完整的例子......

I created my own class based on QWidget. It has a slot that accepts a string; I define it like this:

我基于 QWidget 创建了自己的类。它有一个接受字符串的插槽；我是这样定义的：

@QtCore.pyqtSlot(str)
def add_text(self, text):
   ...

Later, I create an instance of this widget in the main GUI thread. From the main GUI thread or any other thread (knock on wood) I can call:

后来，我在主 GUI 线程中创建了这个小部件的一个实例。从主 GUI 线程或任何其他线程（敲木头）我可以调用：

QtCore.QMetaObject.invokeMethod(mywidget, "add_text", QtCore.Q_ARG(str,"hello world"))

Clunky, but it gets you there.

笨重，但它让你到达那里。

Dan.

担。

回答by Eli Bendersky

I recommend you to use Queue instead of signaling. Personally I find it a much more robust and understandable way of programming, because it's more synchronous.

我建议您使用 Queue 而不是信令。就我个人而言，我发现它是一种更加健壮和易于理解的编程方式，因为它更加同步。

Threads should get "jobs" from a Queue, and put back results on another Queue. Yet a third Queue can be used by the threads for notifications and messages, like errors and "progress reports". Once you structure your code this way, it becomes much simpler to manage.

线程应该从队列中获取“作业”，并将结果放回另一个队列。然而，线程可以使用第三个队列来发送通知和消息，例如错误和“进度报告”。以这种方式构建代码后，管理起来就简单多了。

This way, a single "job Queue" and "result Queue" can also be used by a group of worker threads, it routes all the information from the threads into the main GUI thread.

这样，一组工作线程也可以使用单个“作业队列”和“结果队列”，它将所有信息从线程路由到主 GUI 线程。

回答by zihotki

If your method "processDoc" doesn't change any other data (just looks for some data and return it and don't change variables or properties of parent class) you may use Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macroses ( see here for details) in it. So the document will be processed in thread which will not lock the interpreter and UI will be updated.

如果您的方法“processDoc”不更改任何其他数据（只是查找一些数据并返回它而不更改父类的变量或属性），您可以在其中使用 Py_BEGIN_ALLOW_THREADS 和 Py_END_ALLOW_THREADS 宏（有关详细信息，请参见此处）。因此，文档将在不会锁定解释器的线程中处理，并且 UI 将被更新。

回答by MrEvil

You are always going to have this problem in Python. Google GIL "global interpretor lock" for more background. There are two generally recommended ways to get around the problem that you are experiencing: use Twisted, or use a module similar to the multiprocessingmodule introduced in 2.5.

在 Python 中你总是会遇到这个问题。谷歌 GIL“全局解释器锁”了解更多背景信息。有两种普遍推荐的方法可以解决您遇到的问题：使用Twisted，或使用类似于2.5 中引入的多处理模块的模块。

Twisted will require that you learn asynchronous programming techniques which may be confusing in the beginning but will be helpful if you ever need to write high throughput network apps and will be more beneficial to you in the long run.

Twisted 将要求您学习异步编程技术，这在开始时可能会令人困惑，但如果您需要编写高吞吐量的网络应用程序，这将很有帮助，并且从长远来看对您更有益。

The multiprocessing module will fork a new process and uses IPC to make it behave as if you had true threading. Only downside is that you would need python 2.5 installed which is fairly new and inst' included in most Linux distros or OSX by default.

多处理模块将派生一个新进程并使用 IPC 使其表现得就像您拥有真正的线程一样。唯一的缺点是你需要安装 python 2.5，它是相当新的并且默认包含在大多数 Linux 发行版或 OSX 中。