Python:等待所有`concurrent.futures.ThreadPoolExecutor`的期货

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/21143162/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-18 22:10:20  来源:igfitidea点击:

Python: Wait on all of `concurrent.futures.ThreadPoolExecutor`'s futures

pythonconcurrencyfuture

提问by Ram Rachum

I've given concurrent.futures.ThreadPoolExecutora bunch of tasks, and I want to wait until they're all completed before proceeding with the flow. How can I do that, without having to save all the futures and call waiton them? (I want an action on the executor.)

我给concurrent.futures.ThreadPoolExecutor了一堆任务,我想等到它们都完成后再继续流程。我怎样才能做到这一点,而不必保存所有期货并调用wait它们?(我想对执行者采取行动。)

采纳答案by Bakuriu

Just call Executor.shutdown:

只需致电Executor.shutdown

shutdown(wait=True)

Signal the executor that it should free any resources that it is using when the currently pending futures are done executing. Calls to Executor.submit()and Executor.map()made after shutdown will raise RuntimeError.

If wait is Truethen this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed.

shutdown(wait=True)

当当前挂起的期货完成执行时,向执行器发出信号,它应该释放它正在使用的任何资源。关闭后调用Executor.submit()Executor.map()将引发RuntimeError.

如果 wait 是,True那么这个方法将不会返回,直到所有挂起的期货都执行完毕并且与执行程序关联的资源已被释放。

However if you keep track of your futures in a list then you can avoid shutting the executor down for future use using the futures.wait()function:

但是,如果您在列表中跟踪您的期货,那么您可以避免使用以下futures.wait()功能关闭执行程序以供将来使用:

concurrent.futures.wait(fs, timeout=None, return_when=ALL_COMPLETED)

Wait for the Futureinstances (possibly created by different Executorinstances) given by fsto complete. Returns a named 2-tuple of sets. The first set, named done, contains the futures that completed (finished or were cancelled) before the wait completed. The second set, named not_done, contains uncompleted futures.

concurrent.futures.wait(fs, timeout=None, return_when=ALL_COMPLETED)

等待 给出的Future实例(可能由不同的Executor实例创建 )fs完成。返回一个命名的 2 元组集合。第一个名为 done 的集合包含在等待完成之前完成(完成或被取消)的期货。第二组名为 not_done,包含未完成的期货。

note that if you don't provide a timeoutit waits until all futures have completed.

请注意,如果您不提供 atimeout它会等到所有期货都完成。

You can also use futures.as_completed()instead, however you'd have to iterate over it.

您也可以futures.as_completed()改为使用,但是您必须对其进行迭代。

回答by laike9m

Bakuriu's answer is correct. Just to extend a little bit. As we all know a context manager has __enter__and __exit__method. Here is how class Executor(ThreadPoolExecutor's base class) is defined

巴库留的回答是正确的。只是为了扩展一点。众所周知,上下文管理器具有__enter____exit__方法。这是class ExecutorThreadPoolExecutor 的基类)是如何定义的

class Executor(object):

    # other methods

    def shutdown(self, wait=True):
        """Clean-up the resources associated with the Executor.

        It is safe to call this method several times. Otherwise, no other
        methods can be called after this one.

        Args:
            wait: If True then shutdown will not return until all running
                futures have finished executing and the resources used by the
                executor have been reclaimed.
        """
        pass

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.shutdown(wait=True)
        return False

And it is ThreadPoolExecutorthat actually defines the shutdownmethod

ThreadPoolExecutor实际上定义了shutdown方法

class ThreadPoolExecutor(_base.Executor):
    def shutdown(self, wait=True):
        with self._shutdown_lock:
            self._shutdown = True
            self._work_queue.put(None)
        if wait:
            for t in self._threads:
                t.join()