Python 如何创建一个子进程可以读取的临时文件？

How to create a temporary file that can be read by a subprocess?

提问by Nathan Reed

I'm writing a Python script that needs to write some data to a temporary file, then create a subprocess running a C++ program that will read the temporary file. I'm trying to use NamedTemporaryFilefor this, but according to the docs,

我正在编写一个需要将一些数据写入临时文件的 Python 脚本，然后创建一个运行 C++ 程序的子进程来读取临时文件。我正在尝试NamedTemporaryFile用于此，但根据文档，

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

在命名的临时文件仍处于打开状态时，是否可以使用该名称再次打开该文件，因平台而异（在 Unix 上可以如此使用；在 Windows NT 或更高版本上则不能）。

And indeed, on Windows if I flush the temporary file after writing, but don't close it until I want it to go away, the subprocess isn't able to open it for reading.

事实上，在 Windows 上，如果我在写入后刷新临时文件，但在我希望它消失之前不要关闭它，子进程无法打开它进行读取。

I'm working around this by creating the file with delete=False, closing it before spawning the subprocess, and then manually deleting it once I'm done:

我正在通过使用 来创建文件delete=False，在生成子进程之前关闭它，然后在完成后手动删除它来解决这个问题：

fileTemp = tempfile.NamedTemporaryFile(delete = False)
try:
    fileTemp.write(someStuff)
    fileTemp.close()
    # ...run the subprocess and wait for it to complete...
finally:
    os.remove(fileTemp.name)

This seems inelegant. Is there a better way to do this? Perhaps a way to open up the permissions on the temporary file so the subprocess can get at it?

这看起来很不雅观。有一个更好的方法吗？也许是一种打开临时文件权限的方法，以便子进程可以获取它？

采纳答案by Nilanjan Basu

At least if you open a temporary file using existing Python libraries, accessing it from multiple processes is not possible in case of Windows. According to MSDNyou can specify a 3rd parameter (dwSharedMode) shared mode flag FILE_SHARE_READto CreateFile()function which:

至少如果您使用现有的 Python 库打开一个临时文件，在 Windows 的情况下是不可能从多个进程访问它的。根据MSDN，您可以指定第三个参数 ( dwSharedMode) 共享模式标志FILE_SHARE_READ来CreateFile()运行：

Enables subsequent open operations on a file or device to request read access. Otherwise, other processes cannot open the file or device if they request read access. If this flag is not specified, but the file or device has been opened for read access, the function fails.

启用对文件或设备的后续打开操作以请求读取访问。否则，如果其他进程请求读取访问权限，它们将无法打开文件或设备。如果未指定此标志，但已打开文件或设备进行读取访问，则该函数失败。

So, you can write a Windows specific C routine to create a custom temporary file opener function, call it from Python and then you can make your sub-process access the file without any error. But I think you should stick with your existing approach as it is the most portable version and will work on any system and thus is the most elegant implementation.

因此，您可以编写一个 Windows 特定的 C 例程来创建一个自定义的临时文件打开器函数，从 Python 调用它，然后您可以让您的子进程访问该文件而不会出现任何错误。但我认为你应该坚持你现有的方法，因为它是最便携的版本，可以在任何系统上运行，因此是最优雅的实现。

• Discussion on Linux and windows file locking can be found here.

• 可以在此处找到有关 Linux 和 Windows 文件锁定的讨论。

EDIT: Turns out it is possible to open & read the temporary file from multiple processes in Windows too. See Piotr Dobrogost's answer.

编辑：原来也可以从 Windows 中的多个进程打开和读取临时文件。请参阅 Piotr Dobrogost 的回答。

回答by tshepang

You can always go low-level, though am not sure if it's clean enough for you:

你总是可以去低级，但我不确定它是否对你来说足够干净：

fd, filename = tempfile.mkstemp()
try:
    os.write(fd, someStuff)
    os.close(fd)
    # ...run the subprocess and wait for it to complete...
finally:
    os.remove(filename)

回答by Piotr Dobrogost

Accordingto Richard Oudkerk

根据理查德·奥德克（Richard Oudkerk）的说法

(...) the only reason that trying to reopen a NamedTemporaryFilefails on Windows is because when we reopen we need to use O_TEMPORARY.

(...)NamedTemporaryFile在 Windows 上尝试重新打开失败的唯一原因是因为当我们重新打开时我们需要使用O_TEMPORARY.

and he gives an example of how to do this in Python 3.3+

他给出了如何在 Python 3.3+ 中执行此操作的示例

import os, tempfile

DATA = b"hello bob"

def temp_opener(name, flag, mode=0o777):
    return os.open(name, flag | os.O_TEMPORARY, mode)

with tempfile.NamedTemporaryFile() as f:
    f.write(DATA)
    f.flush()
    with open(f.name, "rb", opener=temp_opener) as f:
        assert f.read() == DATA

assert not os.path.exists(f.name)

Because there's no openerparameter in the built-in open()in Python 2.x, we have to combine lower level os.open()and os.fdopen()functions to achieve the same effect:

因为Python 2.xopener的内置没有参数，所以open()我们必须结合底层os.open()和os.fdopen()函数来达到同样的效果：

import subprocess
import tempfile

DATA = b"hello bob"

with tempfile.NamedTemporaryFile() as f:
    f.write(DATA)
    f.flush()

    subprocess_code = \
    """import os
       f = os.fdopen(os.open(r'{FILENAME}', os.O_RDWR | os.O_BINARY | os.O_TEMPORARY), 'rb')
       assert f.read() == b'{DATA}'
    """.replace('\n', ';').format(FILENAME=f.name, DATA=DATA)

    subprocess.check_output(['python', '-c', subprocess_code]) == DATA

回答by Corbin

Since nobody else appears to be interested in leaving this information out in the open...

由于似乎没有其他人有兴趣将这些信息公开......

tempfiledoes expose a function, mkdtemp(), which can trivialize this problem:

tempfile确实暴露了一个函数，mkdtemp()，它可以使这个问题变得微不足道：

try:
    temp_dir = mkdtemp()
    temp_file = make_a_file_in_a_dir(temp_dir)
    do_your_subprocess_stuff(temp_file)
    remove_your_temp_file(temp_file)
finally:
    os.rmdir(temp_dir)

I leave the implementation of the intermediate functions up to the reader, as one might wish to do things like use mkstemp()to tighten up the security of the temporary file itself, or overwrite the file in-place before removing it. I don't particularly know what security restrictions one might have that are not easily planned for by perusing the source of tempfile.

我将中间函数的实现留给读者，因为人们可能希望做一些事情mkstemp()来加强临时文件本身的安全性，或者在删除文件之前就地覆盖文件。我不是特别知道人们可能有哪些安全限制，通过仔细阅读tempfile.

Anyway, yes, using NamedTemporaryFileon Windows might be inelegant, and my solution here might also be inelegant, but you've already decided that Windows support is more important than elegant code, so you might as well go ahead and do something readable.

无论如何，是的，NamedTemporaryFile在 Windows 上使用可能不优雅，我在这里的解决方案也可能不优雅，但是您已经确定 Windows 支持比优雅的代码更重要，因此您不妨继续做一些可读的事情。