使用 os.pipe 和 os.fork() 问题的 Python 程序
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/871447/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Python program using os.pipe and os.fork() issue
提问by Paradox
I've recently needed to write a script that performs an os.fork()to split into two processes. The child process becomes a server process and passes data back to the parent process using a pipe created with os.pipe(). The child closes the 'r'end of the pipe and the parent closes the 'w'end of the pipe, as usual. I convert the returns from pipe() into file objects with os.fdopen.
我最近需要编写一个脚本来执行os.fork()以拆分为两个进程。子进程成为服务器进程并使用由os.pipe()创建的管道将数据传回父进程。像往常一样，孩子关闭'r'管道的末端，父母关闭'w'管道的末端。我使用os.fdopen将 pipe() 的返回值转换为文件对象。
The problem I'm having is this: The process successfully forks, and the child becomes a server. Everything works great and the child dutifully writes data to the open 'w'end of the pipe. Unfortunately the parent end of the pipe does two strange things:
A) It blocks on the read()operation on the 'r'end of the pipe.
Secondly, it fails to read any data that was put on the pipe unless the 'w'end is entirely closed.
我遇到的问题是：进程成功分叉，子进程成为服务器。一切正常，孩子尽职尽责地将数据写入'w'管道的开口端。不幸的是，管道的父端做了两件奇怪的事情：
A）它阻塞了管道末端的read()操作'r'。
其次，除非'w'末端完全关闭，否则它无法读取放在管道上的任何数据。
I immediately thought that buffering was the problem and added pipe.flush()calls, but these didn't help.
我立即认为缓冲是问题所在，并添加了pipe.flush()调用，但这些都没有帮助。
Can anyone shed some light on why the data doesn't appear until the writing end is fully closed? And is there a strategy to make the read()call non blocking?
任何人都可以解释为什么在写入端完全关闭之前数据不会出现？是否有使read()呼叫非阻塞的策略？
This is my first Python program that forked or used pipes, so forgive me if I've made a simple mistake.
这是我第一个分叉或使用管道的 Python 程序，所以如果我犯了一个简单的错误，请原谅我。
回答by Brian
Are you using read() without specifying a size, or treating the pipe as an iterator (for line in f)? If so, that's probably the source of your problem - read() is defined to read until the end of the file before returning, rather than just read what is available for reading. That will mean it will block until the child calls close().
您是在不指定大小的情况下使用 read() 还是将管道视为迭代器 ( for line in f)？如果是这样，那可能是您的问题的根源 - read() 被定义为在返回之前读取到文件末尾，而不仅仅是读取可供读取的内容。这意味着它将阻塞，直到孩子调用 close()。
In the example code linked to, this is OK - the parent is acting in a blocking manner, and just using the child for isolation purposes. If you want to continue, then either use non-blocking IO as in the code you posted (but be prepared to deal with half-complete data), or read in chunks (eg r.read(size) or r.readline()) which will block only until a specific size / line has been read. (you'll still need to call flush on the child)
在链接到的示例代码中，这是可以的 - 父级以阻塞方式运行，并且仅将子级用于隔离目的。如果你想继续，那么要么在你发布的代码中使用非阻塞 IO（但准备处理半完整的数据），要么分块读取（例如 r.read(size) 或 r.readline() ) 只会阻塞到特定大小/行被读取。（您仍然需要对孩子调用flush）
It looks like treating the pipe as an iterator is using some further buffer as well, for "for line in r:" may not give you what you want if you need each line to be immediately consumed. It may be possible to disable this, but just specifying 0 for the buffer size in fdopen doesn't seem sufficient.
看起来将管道视为迭代器也在使用一些进一步的缓冲区，因为for line in r:如果您需要立即使用每一行，“ ”可能不会给您想要的东西。可以禁用此功能，但仅将 fdopen 中的缓冲区大小指定为 0 似乎还不够。
Heres some sample code that should work:
下面是一些应该可以工作的示例代码：
import os, sys, time
r,w=os.pipe()
r,w=os.fdopen(r,'r',0), os.fdopen(w,'w',0)
pid = os.fork()
if pid: # Parent
w.close()
while 1:
data=r.readline()
if not data: break
print "parent read: " + data.strip()
else: # Child
r.close()
for i in range(10):
print >>w, "line %s" % i
w.flush()
time.sleep(1)
回答by Paradox
Using
使用
fcntl.fcntl(readPipe, fcntl.F_SETFL, os.O_NONBLOCK)
fcntl.fcntl(readPipe, fcntl.F_SETFL, os.O_NONBLOCK)
Before invoking the read() solved both problems. The read() call is no longer blocking and the data is appearing after just a flush() on the writing end.
在调用 read() 之前解决了这两个问题。read() 调用不再阻塞，并且数据仅在写入端的 flush() 之后出现。
回答by dF.
I see you have solved the problem of blocking i/o and buffering.
我看到你已经解决了阻塞 i/o 和缓冲的问题。
A note if you decide to try a different approach: subprocess is the equivalent / a replacement for the fork/exec idiom. It seems like that's not what you're doing: you have just a fork (not an exec) and exchanging data between the two processes -- in this case the multiprocessingmodule (in Python 2.6+) would be a better fit.
如果您决定尝试不同的方法，请注意： subprocess 是 fork/exec 惯用语的等效/替代品。这似乎不是你在做什么：你只有一个 fork（不是 exec）并在两个进程之间交换数据——在这种情况下，multiprocessing模块（在 Python 2.6+ 中）会更合适。
回答by S.Lott
The "parent" vs. "child" part of fork in a Python application is silly. It's a legacy from 16-bit unix days. It's an affectation from a day when fork/exec and exec were Important Things to make the most of a tiny little processor.
Python 应用程序中 fork 的“父”与“子”部分是愚蠢的。这是 16 位 unix 时代的遗产。这是在 fork/exec 和 exec 是充分利用微型处理器的重要事情的那一天的矫揉造作。
Break your Python code into two separate parts: parent and child.
将您的 Python 代码分成两个独立的部分：父代码和子代码。
The parent part should use subprocessto run the child part.
父部件应使用子流程来运行子部件。
A fork and exec may happen somewhere in there -- but you don't need to care.
fork 和 exec 可能发生在那里的某个地方——但你不需要关心。
