Python 子进程 Popen.communicate() 等价于 Popen.stdout.read()？

Python subprocess Popen.communicate() equivalent to Popen.stdout.read()?

提问by Christophe

Very specific question (I hope): What are the differences between the following three codes?

非常具体的问题（我希望）：以下三个代码之间有什么区别？

(I expect it to be only that the first does not wait for the child process to be finished, while the second and third ones do. But I need to be sure this is the onlydifference...)

（我希望只是第一个不等待子进程完成，而第二个和第三个则这样做。但我需要确保这是唯一的区别......）

I also welcome other remarks/suggestions (though I'm already well aware of the shell=Truedangers and cross-platform limitations)

我也欢迎其他评论/建议（尽管我已经很清楚shell=True危险和跨平台限制）

Note that I already read Python subprocess interaction, why does my process work with Popen.communicate, but not Popen.stdout.read()?and that I do not want/need to interact with the program after.

请注意，我已经阅读了Python 子进程交互，为什么我的进程可以使用 Popen.communicate 而不是 Popen.stdout.read()？并且我不想/不需要在之后与程序进行交互。

Also note that I already read Alternatives to Python Popen.communicate() memory limitations?but that I didn't really get it...

另请注意，我已经阅读了 Python Popen.communicate() 内存限制的替代方案？但我并没有真正理解它......

Finally, note that I am aware that somewhere there is a risk of deadlock when one buffer is filled with one output using one method, but I got lost while looking for clear explanations on the Internet...

最后，请注意，我知道当使用一种方法用一个输出填充一个缓冲区时，某处存在死锁的风险，但是我在互联网上寻找明确的解释时迷路了......

First code:

第一个代码：

from subprocess import Popen, PIPE

def exe_f(command='ls -l', shell=True):
    """Function to execute a command and return stuff"""

    process = Popen(command, shell=shell, stdout=PIPE, stderr=PIPE)

    stdout = process.stdout.read()
    stderr = process.stderr.read()

    return process, stderr, stdout

Second code:

第二个代码：

from subprocess import Popen, PIPE
from subprocess import communicate

def exe_f(command='ls -l', shell=True):
    """Function to execute a command and return stuff"""

    process = Popen(command, shell=shell, stdout=PIPE, stderr=PIPE)

    (stdout, stderr) = process.communicate()

    return process, stderr, stdout

Third code:

第三个代码：

from subprocess import Popen, PIPE
from subprocess import wait

def exe_f(command='ls -l', shell=True):
    """Function to execute a command and return stuff"""

    process = Popen(command, shell=shell, stdout=PIPE, stderr=PIPE)

    code   = process.wait()
    stdout = process.stdout.read()
    stderr = process.stderr.read()

    return process, stderr, stdout

Thanks.

谢谢。

采纳答案by jdi

If you look at the source for subprocess.communicate(), it shows a perfect example of the difference:

如果您查看 的来源subprocess.communicate()，它显示了差异的完美示例：

def communicate(self, input=None):
    ...
    # Optimization: If we are only using one pipe, or no pipe at
    # all, using select() or threads is unnecessary.
    if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
        stdout = None
        stderr = None
        if self.stdin:
            if input:
                self.stdin.write(input)
            self.stdin.close()
        elif self.stdout:
            stdout = self.stdout.read()
            self.stdout.close()
        elif self.stderr:
            stderr = self.stderr.read()
            self.stderr.close()
        self.wait()
        return (stdout, stderr)

    return self._communicate(input)

You can see that communicatedoes make use of the read calls to stdoutand stderr, and also calls wait(). It is just a matter of order of operations. In your case because you are using PIPEfor both stdout and stderr, it goes into _communicate():

您可以看到它communicate确实利用了对stdoutand的 read 调用stderr，并且还调用了wait(). 这只是操作顺序的问题。在您的情况下，因为您同时使用PIPEstdout 和 stderr，它进入_communicate()：

def _communicate(self, input):
    stdout = None # Return
    stderr = None # Return

    if self.stdout:
        stdout = []
        stdout_thread = threading.Thread(target=self._readerthread,
                                         args=(self.stdout, stdout))
        stdout_thread.setDaemon(True)
        stdout_thread.start()
    if self.stderr:
        stderr = []
        stderr_thread = threading.Thread(target=self._readerthread,
                                         args=(self.stderr, stderr))
        stderr_thread.setDaemon(True)
        stderr_thread.start()

    if self.stdin:
        if input is not None:
            self.stdin.write(input)
        self.stdin.close()

    if self.stdout:
        stdout_thread.join()
    if self.stderr:
        stderr_thread.join()

    # All data exchanged.  Translate lists into strings.
    if stdout is not None:
        stdout = stdout[0]
    if stderr is not None:
        stderr = stderr[0]

    # Translate newlines, if requested.  We cannot let the file
    # object do the translation: It is based on stdio, which is
    # impossible to combine with select (unless forcing no
    # buffering).
    if self.universal_newlines and hasattr(file, 'newlines'):
        if stdout:
            stdout = self._translate_newlines(stdout)
        if stderr:
            stderr = self._translate_newlines(stderr)

    self.wait()
    return (stdout, stderr)

This uses threads to read from multiple streams at once. Then it calls wait()at the end.

这使用线程一次从多个流中读取。然后它wait()在最后调用。

So to sum it up:

所以总结一下：

1. This example reads from one stream at a time and does not wait for it to finish the process.
2. This example reads from both streams at the same time via internal threads, and waits for it to finish the process.
3. This example waits for the process to finish, and then reads one stream at a time. And as you mentioned has the potential to deadlock if there is too much written to the streams.

1. 此示例一次从一个流中读取，并且不等待它完成该过程。
2. 此示例通过内部线程同时从两个流读取，并等待它完成该过程。
3. 此示例等待进程完成，然后一次读取一个流。正如您所提到的，如果写入流的内容过多，则可能会出现死锁。

Also, you don't need these two import statements in your 2nd and 3rd examples:

此外，您在第二个和第三个示例中不需要这两个导入语句：

from subprocess import communicate
from subprocess import wait

They are both methods of the Popenobject.

它们都是Popen对象的方法。