Python yield 如何捕获 StopIteration 异常？

How yield catches StopIteration exception?

提问by Sergey Ivanov

Why in the example function terminates:

为什么在示例函数中终止：

def func(iterable):
    while True:
        val = next(iterable)
        yield val

but if I take off yield statement function will raise StopIteration exception?

但是如果我去掉 yield 语句函数会引发 StopIteration 异常吗？

EDIT:Sorry for misleading you guys. I know what generators are and how to use them. Of course when I said function terminates I didn't mean eager evaluation of function. I just implied that when I use function to produce generator:

编辑：抱歉误导你们。我知道什么是生成器以及如何使用它们。当然，当我说函数终止时，我并不是指对函数的急切评估。我只是暗示当我使用函数来生成生成器时：

gen = func(iterable)

in case of funcit works and returns the same generator, but in case of func2:

在func 的情况下，它可以工作并返回相同的生成器，但在 func2 的情况下：

def func2(iterable):
    while True:
        val = next(iterable)

it raises StopIteration instead of Nonereturn or infinite loop.

它引发 StopIteration 而不是None返回或无限循环。

Let me be more specific. There is a function teein itertoolswhich is equivalent to:

让我说得更具体一点。还有一个功能发球在itertools这相当于：

def tee(iterable, n=2):
    it = iter(iterable)
    deques = [collections.deque() for i in range(n)]
    def gen(mydeque):
        while True:
            if not mydeque:             # when the local deque is empty
                newval = next(it)       # fetch a new value and
                for d in deques:        # load it to all the deques
                    d.append(newval)
            yield mydeque.popleft()
    return tuple(gen(d) for d in deques)

There is, in fact, some magic, because nested function genhas infinite loop without break statements. genfunction terminates due to StopIterationexception when there is no items in it. But it terminates correctly (without raising exceptions), i.e. just stops loop. So the question is: where is StopIterationis handled?

事实上，有一些神奇之处，因为嵌套函数gen具有没有 break 语句的无限循环。根函数结束因的StopIteration异常时，有中没有的项目它。但它正确终止（不引发异常），即只是停止循环。所以问题是：在哪里处理StopIteration？

采纳答案by Blckknght

To answer your question about where the StopIterationgets caught in the gengenerator created inside of itertools.tee: it doesn't. It is up to the consumer of the teeresults to catch the exception as they iterate.

回答您关于在内部创建StopIteration的gen生成器中捕获的位置的问题itertools.tee：它没有。tee结果的使用者在迭代时捕获异常。

First off, it's important to note that a generator function (which is any function with a yieldstatement in it, anywhere) is fundamentally different than a normal function. Instead of running the function's code when it is called, instead, you'll just get a generatorobject when you call the function. Only when you iterate over the generator will you run the code.

首先，重要的是要注意生成器函数（它是任何带有yield语句的函数，任何地方）与普通函数根本不同。不是在调用函数时运行函数的代码，而是generator在调用函数时获得一个对象。只有当您迭代生成器时，您才会运行代码。

A generator function will never finish iterating without raising StopIteration(unless it raises some other exception instead). StopIterationis the signal from the generator that it is done, and it is not optional. If you reach a returnstatement or the end of the generator function's code without raising anything, Python will raise StopIterationfor you!

生成器函数永远不会在不引发的情况下完成迭代StopIteration（除非它引发了其他一些异常）。StopIteration是来自生成器的信号，表示它已完成，它不是可选的。如果你到达一个return语句或生成器函数代码的末尾而没有引发任何事情，Python 会StopIteration为你引发！

This is different from regular functions, which return Noneif they reach the end without returning anything else. It ties in with the different ways that generators work, as I described above.

这与常规函数不同，常规函数None在到达结束时返回而不返回任何其他内容。正如我上面所描述的，它与生成器的不同工作方式有关。

Here's an example generator function that will make it easy to see how StopIterationgets raised:

这是一个示例生成器函数，它可以让您轻松了解如何StopIteration引发：

def simple_generator():
    yield "foo"
    yield "bar"
    # StopIteration will be raised here automatically

Here's what happens when you consume it:

以下是您食用它时会发生的情况：

>>> g = simple_generator()
>>> next(g)
'foo'
>>> next(g)
'bar'
>>> next(g)
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    next(g)
StopIteration

Calling simple_generatoralways returns a generatorobject immediately (without running any of the code in the function). Each call of nexton the generator object runs the code until the next yieldstatement, and returns the yielded value. If there is no more to get, StopIterationis raised.

调用simple_generator总是generator立即返回一个对象（不运行函数中的任何代码）。next对生成器对象的每次调用都会运行代码直到下一条yield语句，并返回产生的值。如果没有更多的东西，StopIteration则提出。

Now, normally you don't see StopIterationexceptions. The reason for this is that you usually consume generators inside forloops. A forstatement will automatically call nextover and over until StopIterationgets raised. It will catch and suppress the StopIterationexception for you, so you don't need to mess around with try/exceptblocks to deal with it.

现在，通常您看不到StopIteration异常。这样做的原因是您通常在for循环内使用生成器。一个for语句会自动next一遍又一遍地调用，直到StopIteration被提出。它会StopIteration为你捕获并抑制异常，所以你不需要用try/except块来处理它。

A forloop like for item in iterable: do_suff(item)is almost exactly equivalent to this whileloop (the only difference being that a real fordoesn't need a temporary variable to hold the iterator):

一个for循环 likefor item in iterable: do_suff(item)几乎完全等同于这个while循环（唯一的区别是 realfor不需要临时变量来保存迭代器）：

iterator = iter(iterable)
try:
    while True:
        item = next(iterator)
        do_stuff(item)
except StopIteration:
    pass
finally:
    del iterator

The gengenerator function you showed at the top is one exception. It uses the StopIterationexception produced by the iterator it is consuming as it's own signal that it is done being iterated on. That is, rather than catching the StopIterationand then breaking out of the loop, it simply lets the exception go uncaught (presumably to be caught by some higher level code).

gen您在顶部显示的生成器函数是一个例外。它使用StopIteration它正在消耗的迭代器产生的异常作为它自己的信号，它已经完成迭代。也就是说，它不是捕获StopIteration然后跳出循环，而是简单地让异常不被捕获（大概会被一些更高级别的代码捕获）。

Unrelated to the main question, there is one other thing I want to point out. In your code, you're calling nexton an variable called iterable. If you take that name as documentation for what type of object you will get, this is not necessarily safe.

与主要问题无关，我想指出另一件事。在您的代码中，您正在调用next一个名为iterable. 如果您将该名称作为您将获得的对象类型的文档，则这不一定是安全的。

nextis part of the iteratorprotocol, not the iterable(or container) protocol. It may work for some kinds of iterables (such as files and generators, as those types are their own iterators), but it will fail for others iterables, such as tuples and lists. The more correct approach is to call iteron your iterablevalue, then call nexton the iterator you receive. (Or just use forloops, which call both iterand nextfor you at appropriate times!)

next是iterator协议的一部分，而不是iterable（或container）协议。它可能适用于某些类型的迭代器（例如文件和生成器，因为这些类型是它们自己的迭代器），但对于其他迭代器，例如元组和列表，它会失败。更正确的方法是调用iter您的iterable值，然后调用next您收到的迭代器。（或者只是使用for循环，它会在适当的时候为您调用iter和调用next！）

Edit: I just found my own answer in a Google search for a related question, and I thought I'd update to point out that the answer above will not be completely true in future Python versions. PEP 479is making it an error to allow a StopIterationto bubble up uncaught from a generator function. If that happens, Python will turn it into a RuntimeErrorexception instead.

编辑：我刚刚在 Google 搜索相关问题中找到了自己的答案，我想我会更新以指出上述答案在未来的 Python 版本中不会完全正确。PEP 479使得允许 aStopIteration冒泡而未从生成器函数中捕获是一个错误。如果发生这种情况，Python 会将其转为RuntimeError异常。

This means that code like the examples in itertoolsthat use a StopIterationto break out of a generator function will need to be modified. Usually you'll need to catch the exception with a try/exceptand then do return.

这意味着像示例中itertools使用 aStopIteration突破生成器函数的代码将需要修改。通常，您需要使用try/捕获异常except，然后执行return.

Because this is a backwards incompatible change, it's being phased in gradually. In Python 3.5, all code will work as before by default, but you can get the new behavior with from __future__ import generator_stop. In Python 3.6, the code will still work, but it will give a warning. In Python 3.7, the new behavior will apply all the time.

因为这是一个向后不兼容的更改，所以它正在逐步实施。在 Python 3.5 中，默认情况下所有代码都将像以前一样工作，但您可以使用from __future__ import generator_stop. 在 Python 3.6 中，代码仍然可以工作，但会发出警告。在 Python 3.7 中，新行为将一直适用。

回答by chepner

Without the yield, you iterate over the entire iterablewithout stopping to do anything with val. The whileloop does not catch the StopIterationexception. An equivalent forloop would be:

如果没有yield，您将在整个过程中进行迭代，而iterable不会停止对 进行任何操作val。该while循环不会赶上StopIteration例外。一个等效的for循环是：

def func(iterable):
    for val in iterable:
        pass

which does catch the StopIterationand simply exit the loop and thus return from the function.

它确实捕获StopIteration并简单地退出循环，从而从函数返回。

You can explicitly catch the exception:

您可以显式捕获异常：

def func(iterable):
    while True:
        try:
            val = next(iterable)
        except StopIteration:
            break

回答by kindall

When a function contains yield, calling it does not actually execute anything, it merely creates a generator object. Only iterating over this object will execute the code. So my guess is that you're merely calling the function, which means the function doesn't raise StopIterationbecause it is never being executed.

当一个函数 contains 时yield，调用它实际上并不执行任何操作，它只是创建一个生成器对象。只有迭代这个对象才会执行代码。所以我的猜测是您只是在调用该函数，这意味着该函数不会引发，StopIteration因为它从未被执行过。

Given your function, and an iterable:

给定你的函数和一个可迭代的：

def func(iterable):
    while True:
        val = next(iterable)
        yield val

iterable = iter([1, 2, 3])

This is the wrong way to call it:

这是错误的调用方式：

func(iterable)

This is the right way:

这是正确的方法：

for item in func(iterable):
    # do something with item

You could also store the generator in a variable and call next()on it (or iterate over it in some other way):

您还可以将生成器存储在一个变量中并调用next()它（或以其他方式迭代它）：

gen = func(iterable)
print(next(gen))   # prints 1
print(next(gen))   # prints 2
print(next(gen))   # prints 3
print(next(gen))   # StopIteration

By the way, a better way to write your function is as follows:

顺便说一句，编写函数的更好方法如下：

def func(iterable):
    for item in iterable:
        yield item

Or in Python 3.3 and later:

或者在 Python 3.3 及更高版本中：

def func(iterable):
    yield from iter(iterable)

Of course, real generators are rarely so trivial. :-)

当然，真正的生成器很少如此微不足道。:-)

回答by mgilson

yielddoesn't catch the StopIteration. What yielddoes for your function is it causes it to become a generator function rather than a regular function. Thus, the object returned from the function call is an iterable object (which calculates the next value when you ask it to with the nextfunction (which gets called implicitly by a for loop)). If you leave the yieldstatement out of it, then python executes the entire whileloop right away which ends up exhausting the iterable (if it is finite) and raising StopIterationright when you call it.

yield没有抓住StopIteration. 什么yield做你的功能是使之成为一台发电机的功能，而不是常规的功能。因此，从函数调用返回的对象是一个可迭代对象（当您使用next函数要求它时计算下一个值（它被 for 循环隐式调用））。如果您将yield语句排除在外，python 会立即执行整个while循环，最终耗尽可迭代对象（如果它是有限的）并StopIteration在您调用它时立即引发。

consider:

考虑：

x = func(x for x in [])
next(x)  #raises StopIteration

A forloop catches the exception -- That's how it knows when to stop calling nexton the iterable you gave it.

一个for循环捕捉异常-这就是它如何知道何时停止调用next你给它的迭代。

回答by erandac

Tested on Python 3.8, chunk as lazy generator

在 Python 3.8 上测试，块作为惰性生成器

def split_to_chunk(size: int, iterable: Iterable) -> Iterable[Iterable]:
    source_iter = iter(iterable)
    while True:
        batch_iter = itertools.islice(source_iter, size)
        try:
            yield itertools.chain([next(batch_iter)], batch_iter)
        except StopIteration:
            return

Why handling StopInteration error: https://www.python.org/dev/peps/pep-0479/

为什么处理 StopInteration 错误：https://www.python.org/dev/peps/pep-0479/

def sample_gen() -> Iterable[int]:
    i = 0
    while True:
        yield i
        i += 1

for chunk in split_to_chunk(7, sample_gen()):
    pprint.pprint(list(chunk))
    time.sleep(2)

Output:

输出：

[0, 1, 2, 3, 4, 5, 6]
[7, 8, 9, 10, 11, 12, 13]
[14, 15, 16, 17, 18, 19, 20]
[21, 22, 23, 24, 25, 26, 27]
............................