Python 如何进行虚拟文件处理?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/18550127/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to do virtual file processing?
提问by Steve Grafton
So for creating files I use the following:
因此,对于创建文件,我使用以下内容:
fileHandle = open('fileName', 'w')
then write the contents to the file, close the file. In the next step I process the file. At the end of the program, I end up with a "physical file" that I need to delete.
然后将内容写入文件,关闭文件。在下一步中,我处理该文件。在程序结束时,我最终得到一个需要删除的“物理文件”。
Is there a way to write a "virtual" file that behaves exactly like a "physical" one (allowing it to be manipulated the same way) but does not exist at the end of the run in Python?
有没有办法编写一个“虚拟”文件,它的行为与“物理”文件完全一样(允许以相同的方式操作)但在 Python 运行结束时不存在?
采纳答案by martineau
You might want to consider using a tempfile.SpooledTemporaryFile
which gives you the best of both worlds in the sense that it will create a temporary memory-based virtual file initially but will automatically switch to a physical disk-based file if the data held in memory exceeds a specified size.
您可能需要考虑使用 a tempfile.SpooledTemporaryFile
,它为您提供两全其美的方式,因为它最初会创建一个基于内存的临时虚拟文件,但如果内存中保存的数据超过指定的数量,则会自动切换到基于物理磁盘的文件尺寸。
Another nice feature is that (when using memory) it will automatically use either an io.BytesIO
or io.StringIO
depending on what mode
is being used—allowing you to either read and write Unicode strings or binary data (bytes) to it.
另一个不错的功能是(在使用内存时)它会根据正在使用的内容自动使用io.BytesIO
或- 允许您读取和写入 Unicode 字符串或二进制数据(字节)。io.StringIO
mode
The only tricky part might be the fact that you'll need to avoid closing the file between steps because doing so would cause it to be deleted from memory or disk. Instead you can just rewind it back to the beginning with a file seek(0)
method call.
唯一棘手的部分可能是您需要避免在步骤之间关闭文件,因为这样做会导致它从内存或磁盘中删除。相反,您可以使用文件seek(0)
方法调用将其倒回到开头。
When you are completely done with the file and close it, it will automatically be deleted from disk if the amount of data in it caused it to be rolled-over to a physical file.
当您完成文件并关闭它时,如果其中的数据量导致它被滚动到物理文件,它将自动从磁盘中删除。
回答by Viktor Kerkez
You have StringIO
and BytesIO
in the io
module.
你有StringIO
和BytesIO
在io
模块中。
StringIO
behaves like a file opened in text mode - reading and writing unicode strings (equivalent to opening a file with io.open(filename, mode, encoding='...')
), and the BytesIO
behaves like a file opened in binary mode (mode='[rw]b'
), and can read write bytes.
StringIO
行为类似于以文本模式打开的文件 - 读取和写入 unicode 字符串(相当于使用 打开文件io.open(filename, mode, encoding='...')
),BytesIO
行为类似于以二进制模式 ( mode='[rw]b'
)打开的文件,并且可以读取写入字节。
Python 2:
蟒蛇2:
In [4]: f = io.BytesIO('test')
In [5]: type(f.read())
Out[5]: str
In [6]: f = io.StringIO(u'test')
In [7]: type(f.read())
Out[7]: unicode
Python 3:
蟒蛇3:
In [2]: f = io.BytesIO(b'test')
In [3]: type(f.read())
Out[3]: builtins.bytes
In [4]: f = io.StringIO('test')
In [5]: type(f.read())
Out[5]: builtins.str
回答by Srinivas Reddy Thatiparthy
You can use StringIO as a virtual file , from official documentation
您可以使用 StringIO 作为虚拟文件,来自官方文档
from io import StringIO
output = StringIO()
output.write('First line.\n')
print >>output, 'Second line.'
# Retrieve file contents -- this will be
# 'First line.\nSecond line.\n'
contents = output.getvalue()
# Close object and discard memory buffer --
# .getvalue() will now raise an exception.
output.close()
回答by nickie
There is the StringIO
module, read its documentation, it should be easy to use.
有StringIO
模块,阅读它的文档,应该很容易使用。
Bear in mind, though, that this would keep the "file's" contents in memory. If you have too much data, it would probably be better to create a real file, e.g. in /tmp, and delete it afterwards.
但是请记住,这会将“文件”的内容保留在内存中。如果您有太多数据,最好创建一个真实文件,例如在 /tmp 中,然后将其删除。
回答by Apostolos
If you mean writing to memory instead of a file, you can simply write the text to a buffer and use the following function:
如果您的意思是写入内存而不是文件,您可以简单地将文本写入缓冲区并使用以下函数:
def write(text):
global buffer
buffer += text + '\n' # Add a linefeed as you would if you were writing to a file
buffer = "" # Initialize the buffer
write("My name is Steve Grafton")
At the end, you will have a buffer that will be the same as if you had written your stuff to a file and then open the file and read all its contents to a buffer! Moreover, you can use the buffer during the process (before having finished your writing) and do searches in it, as if you had created a file for both reading and writing, only that in this case your pointer will
最后,您将拥有一个缓冲区,就像您将内容写入文件然后打开文件并将其所有内容读取到缓冲区一样!此外,您可以在此过程中(在完成写入之前)使用缓冲区并在其中进行搜索,就像您创建了一个用于读取和写入的文件一样,只有在这种情况下,您的指针才会