Python 3.5 中协程和未来/任务之间的区别？

Question

提问by knite

Let's say we have a dummy function:

假设我们有一个虚拟函数：

async def foo(arg):
    result = await some_remote_call(arg)
    return result.upper()

What's the difference between:

有什么区别：

coros = []
for i in range(5):
    coros.append(foo(i))

loop = get_event_loop()
loop.run_until_complete(wait(coros))

And:

和：

from asyncio import ensure_future

futures = []
for i in range(5):
    futures.append(ensure_future(foo(i)))

loop = get_event_loop()
loop.run_until_complete(wait(futures))

Note: The example returns a result, but this isn't the focus of the question. When return value matters, use gather()instead of wait().

注意：该示例返回一个结果，但这不是问题的重点。当返回值很重要时，使用gather()代替wait().

Regardless of return value, I'm looking for clarity on ensure_future(). wait(coros)and wait(futures)both run the coroutines, so when and why should a coroutine be wrapped in ensure_future?

无论返回值如何，我都在寻找ensure_future(). wait(coros)并且wait(futures)都运行协同程序，那么何时以及为什么应该将协同程序包装在ensure_future？

Basically, what's the Right Way (tm) to run a bunch of non-blocking operations using Python 3.5's async?

基本上，使用 Python 3.5 运行一堆非阻塞操作的正确方法 (tm) 是async什么？

For extra credit, what if I want to batch the calls? For example, I need to call some_remote_call(...)1000 times, but I don't want to crush the web server/database/etc with 1000 simultaneous connections. This is doable with a thread or process pool, but is there a way to do this with asyncio?

为了获得额外的积分，如果我想批量调用怎么办？例如，我需要调用some_remote_call(...)1000 次，但我不想用 1000 个同时连接来粉碎 Web 服务器/数据库/等。这对于线程或进程池是可行的，但是有没有办法做到这一点asyncio？

Answer 1

采纳答案by knite

A comment by Vincent linked to https://github.com/python/asyncio/blob/master/asyncio/tasks.py#L346, which shows that wait()wraps the coroutines in ensure_future()for you!

Vincent 的评论链接到https://github.com/python/asyncio/blob/master/asyncio/tasks.py#L346，这表明为您wait()包装了协程ensure_future()！

In other words, we do need a future, and coroutines will be silently transformed into them.

换句话说，我们确实需要一个未来，协程会默默地转化成他们。

I'll update this answer when I find a definitive explanation of how to batch coroutines/futures.

当我找到有关如何批处理协程/期货的明确解释时，我将更新此答案。

Answer 2

回答by masnun

A coroutine is a generator function that can both yield values and accept values from the outside. The benefit of using a coroutine is that we can pause the execution of a function and resume it later. In case of a network operation, it makes sense to pause the execution of a function while we're waiting for the response. We can use the time to run some other functions.

协程是一个生成器函数，它既可以产生值也可以接受来自外部的值。使用协程的好处是我们可以暂停函数的执行并在稍后恢复它。在网络操作的情况下，在我们等待响应时暂停函数的执行是有意义的。我们可以利用时间来运行一些其他功能。

A future is like the Promiseobjects from Javascript. It is like a placeholder for a value that will be materialized in the future. In the above-mentioned case, while waiting on network I/O, a function can give us a container, a promise that it will fill the container with the value when the operation completes. We hold on to the future object and when it's fulfilled, we can call a method on it to retrieve the actual result.

未来就像Promise来自 Javascript的对象。它就像一个未来将实现的价值的占位符。在上述情况下，在等待网络 I/O 时，函数可以给我们一个容器，承诺在操作完成时将值填充容器。我们持有未来的对象，当它完成时，我们可以调用它的方法来检索实际结果。

Direct Answer:You don't need ensure_futureif you don't need the results. They are good if you need the results or retrieve exceptions occurred.

直接回答：你并不需要ensure_future，如果你不想要的结果。如果您需要结果或检索发生异常，它们会很好。

Extra Credits:I would choose run_in_executorand pass an Executorinstance to control the number of max workers.

额外积分：我会选择run_in_executor并传递一个Executor实例来控制最大工人的数量。

Explanations and Sample codes

说明和示例代码

In the first example, you are using coroutines. The waitfunction takes a bunch of coroutines and combines them together. So wait()finishes when all the coroutines are exhausted (completed/finished returning all the values).

在第一个示例中，您使用的是协程。该wait函数采用一堆协程并将它们组合在一起。所以wait()当所有协程都用完时结束（完成/完成返回所有值）。

loop = get_event_loop() # 
loop.run_until_complete(wait(coros))

The run_until_completemethod would make sure that the loop is alive until the execution is finished. Please notice how you are not getting the results of the async execution in this case.

该run_until_complete方法将确保循环处于活动状态，直到执行完成。请注意在这种情况下您没有获得异步执行的结果。

In the second example, you are using the ensure_futurefunction to wrap a coroutine and return a Taskobject which is a kind of Future. The coroutine is scheduled to be executed in the main event loop when you call ensure_future. The returned future/task object doesn't yet have a value but over time, when the network operations finish, the future object will hold the result of the operation.

在第二个示例中，您使用该ensure_future函数来包装协程并返回一个Task对象，该对象是一种Future. 当您调用ensure_future. 返回的 future/task 对象还没有值，但随着时间的推移，当网络操作完成时，future 对象将保存操作的结果。

from asyncio import ensure_future

futures = []
for i in range(5):
    futures.append(ensure_future(foo(i)))

loop = get_event_loop()
loop.run_until_complete(wait(futures))

So in this example, we're doing the same thing except we're using futures instead of just using coroutines.

所以在这个例子中，我们正在做同样的事情，除了我们使用期货而不是仅仅使用协程。

Let's look at an example of how to use asyncio/coroutines/futures:

我们来看一个如何使用 asyncio/coroutines/futures 的例子：

import asyncio


async def slow_operation():
    await asyncio.sleep(1)
    return 'Future is done!'


def got_result(future):
    print(future.result())

    # We have result, so let's stop
    loop.stop()


loop = asyncio.get_event_loop()
task = loop.create_task(slow_operation())
task.add_done_callback(got_result)

# We run forever
loop.run_forever()

Here, we have used the create_taskmethod on the loopobject. ensure_futurewould schedule the task in the main event loop. This method enables us to schedule a coroutine on a loop we choose.

在这里，我们create_task在loop对象上使用了方法。ensure_future将在主事件循环中安排任务。这种方法使我们能够在我们选择的循环上安排协程。

We also see the concept of adding a callback using the add_done_callbackmethod on the task object.

我们还看到了使用add_done_callback任务对象上的方法添加回调的概念。

A Taskis donewhen the coroutine returns a value, raises an exception or gets canceled. There are methods to check these incidents.

ATask是done当协程返回一个值、引发异常或被取消时。有一些方法可以检查这些事件。

I have written some blog posts on these topics which might help:

我写了一些关于这些主题的博客文章，可能会有所帮助：

Of course, you can find more details on the official manual: https://docs.python.org/3/library/asyncio.html

当然，你可以在官方手册上找到更多细节：https: //docs.python.org/3/library/asyncio.html

Answer 3

回答by crizCraig

From the BDFL [2013]

来自 BDFL [2013]

Tasks

任务

It's a coroutine wrapped in a Future
class Task is a subclass of class Future
So it works with awaittoo!

这是一个包裹在 Future 中的协程
类 Task 是类 Future 的子类
所以它也适用于等待！

How does it differ from a bare coroutine?
It can make progress without waiting for it
- As long as you wait for something else, i.e.
  - await[something_else]

它与裸协程有何不同？
无需等待即可取得进展
- 只要你等待别的东西，即
  - 等待[something_else]

With this in mind, ensure_futuremakes sense as a name for creating a Task since the Future's result will be computed whether or not you awaitit (as long as you await something). This allows the event loop to complete your Task while you're waiting on other things. Note that in Python 3.7 create_taskis the preferred way ensure a future.

考虑到这一点，ensure_future作为创建任务的名称是有意义的，因为无论您是否等待（只要您等待某事），都会计算 Future 的结果。这允许事件循环在您等待其他事情时完成您的任务。请注意，在 Python 3.7 中create_task是确保未来的首选方式。

Note: I changed "yield from" in Guido's slides to "await" here for modernity.

注意：为了现代性，我将 Guido 幻灯片中的“yield from”更改为“await”。

Answer 4

回答by ospider

Simple answer

简单的回答

Invoking a coroutine function(async def) does NOT run it. It returns a coroutine objects, like generator function returns generator objects.
awaitretrieves values from coroutines, i.e. "calls" the coroutine
eusure_future/create_taskschedule the coroutine to run on the event loop on next iteration(although not waiting them to finish, like a daemon thread).

调用协程函数( async def) 不会运行它。它返回一个协程对象，就像生成器函数返回生成器对象一样。
await从协程中检索值，即“调用”协程
eusure_future/create_task安排协程在下一次迭代时在事件循环上运行（尽管不等待它们完成，就像守护线程一样）。

Some code examples

一些代码示例

Let's first clear some terms:

让我们首先明确一些术语：

coroutine function, the one you async defs;
coroutine object, what you got when you "call" a coroutine function;
task, a object wrapped around a coroutine object to run on the event loop.

协程函数，你async def的那个；
协程对象，当你“调用”一个协程函数时你得到了什么；
任务，一个对象包裹在协程对象上以在事件循环上运行。

Case 1, `await`on a coroutine

案例 1，`await`在协程上

We create two coroutines, awaitone, and use create_taskto run the other one.

我们创建了两个协程，await一个create_task用于运行另一个。

import asyncio
import time

# coroutine function
async def p(word):
    print(f'{time.time()} - {word}')


async def main():
    loop = asyncio.get_event_loop()
    coro = p('await')  # coroutine
    task2 = loop.create_task(p('create_task'))  # <- runs in next iteration
    await coro  # <-- run directly
    await task2

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

you will get result:

你会得到结果：

1539486251.7055213 - await
1539486251.7055705 - create_task

Explain:

解释：

task1 was executed directly, and task2 was executed in the following iteration.

task1 直接执行，task2 在接下来的迭代中执行。

Case 2, yielding control to event loop

情况 2，将控制权交给事件循环

If we replace the main function, we can see a different result:

如果我们替换 main 函数，我们可以看到不同的结果：

async def main():
    loop = asyncio.get_event_loop()
    coro = p('await')
    task2 = loop.create_task(p('create_task'))  # scheduled to next iteration
    await asyncio.sleep(1)  # loop got control, and runs task2
    await coro  # run coro
    await task2

you will get result:

你会得到结果：

-> % python coro.py
1539486378.5244057 - create_task
1539486379.5252144 - await  # note the delay

Explain:

解释：

When calling asyncio.sleep(1), the control was yielded back to the event loop, and the loop checks for tasks to run, then it runs the task created by create_task.

调用时asyncio.sleep(1)，控制权被交还给事件循环，循环检查要运行的任务，然后运行由create_task.

Note that, we first invoke the coroutine function, but not awaitit, so we just created a single coroutine, and not make it running. Then, we call the coroutine function again, and wrap it in a create_taskcall, creat_task will actually schedule the coroutine to run on next iteration. So, in the result, create taskis executed before await.

请注意，我们首先调用了协程函数，而不是await它，所以我们只创建了一个协程，并没有让它运行。然后，我们再次调用协程函数，并将其包装在一个create_task调用中，creat_task 实际上会安排协程在下一次迭代中运行。所以，在结果中，create task是在之前执行的await。

Actually, the point here is to give back control to the loop, you could use asyncio.sleep(0)to see the same result.

实际上，这里的重点是将控制权交还给循环，您可以使用它asyncio.sleep(0)来查看相同的结果。

Under the hood

引擎盖下

loop.create_taskactually calls asyncio.tasks.Task(), which will call loop.call_soon. And loop.call_soonwill put the task in loop._ready. During each iteration of the loop, it checks for every callbacks in loop._ready and runs it.

loop.create_task实际上调用asyncio.tasks.Task()，这将调用loop.call_soon。并将loop.call_soon任务放入loop._ready. 在循环的每次迭代期间，它会检查 loop._ready 中的每个回调并运行它。

asyncio.wait, asyncio.ensure_futureand asyncio.gatheractually call loop.create_taskdirectly or indirectly.

asyncio.wait，asyncio.ensure_future和asyncio.gather实际调用loop.create_task直接或间接的影响。

Also note in the docs:

还要注意文档：

Callbacks are called in the order in which they are registered. Each callback will be called exactly once.

回调按注册的顺序调用。每个回调只会被调用一次。

Python 3.5 中协程和未来/任务之间的区别？

提问by knite

采纳答案by knite

回答by masnun

Explanations and Sample codes

说明和示例代码

回答by crizCraig

Tasks

任务

回答by ospider

Simple answer

简单的回答

Some code examples

一些代码示例

Case 1, `await`on a coroutine

案例 1，`await`在协程上

Case 2, yielding control to event loop

情况 2，将控制权交给事件循环

Under the hood

引擎盖下

相关推荐

最近更新

标签

Python 3.5 中协程和未来/任务之间的区别？

提问by knite

采纳答案by knite

回答by masnun

Explanations and Sample codes

说明和示例代码

回答by crizCraig

Tasks

任务

回答by ospider

Simple answer

简单的回答

Some code examples

一些代码示例

Case 1, awaiton a coroutine

案例 1，await在协程上

Case 2, yielding control to event loop

情况 2，将控制权交给事件循环

Under the hood

引擎盖下

相关推荐

Python 在 2 个以上的熊猫数据框中联合

Python 中的 XGBoost XGBClassifier 默认值

Python 字符数组声明

在 Mac OS X 上在 python 2.7 和 python 3.5 之间切换

相关推荐

最近更新

标签

Case 1, `await`on a coroutine

案例 1，`await`在协程上