windows 从 C# 中不断增长的文件中读取?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/1409492/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Read from a growing file in C#?
提问by Robert Fraser
In C#/.NET (on Windows) is there a way to read a "growing" file using a file stream? The length of the file will be very small when the filestream is opened, but the file will be being written to by another thread. If/when the filestream "catches up" to the other thread (i.e. when Read()
returns 0 bytes read), I want to pause to allow the file to buffer a bit, then continue reading.
在 C#/.NET(在 Windows 上)中,有没有办法使用文件流读取“增长”的文件?打开文件流时文件的长度将非常小,但文件将被另一个线程写入。如果/当文件流“赶上”到另一个线程时(即当Read()
返回 0 字节读取时),我想暂停以允许文件缓冲一点,然后继续读取。
I don't really want to use a FilesystemWatcher
and keep creating new file streams (as was suggested for log files), since this isn't a log file (it's a video file being encoded on the fly) and performance is an issue.
我真的不想使用 aFilesystemWatcher
并继续创建新的文件流(如对日志文件的建议),因为这不是日志文件(它是动态编码的视频文件)并且性能是一个问题。
Thanks,
Robert
谢谢,
罗伯特
采纳答案by Vinay Sajip
You can do this, but you need to keep careful track of the file read and write positions using Stream.Seek
and with appropriate synchronization between the threads. Typically you would use an EventWaitHandle
or subclass thereof to do the synchronization for data, and you would also need to consider synchronization for the access to the FileStream
object itself (probably via a lock
statement).
您可以这样做,但您需要使用Stream.Seek
线程之间的适当同步以及适当的同步来仔细跟踪文件读取和写入位置。通常,您会使用一个EventWaitHandle
或它的子类来进行数据同步,并且您还需要考虑对FileStream
对象本身的访问进行同步(可能通过lock
语句)。
Update:In answering this questionI implemented something similar - a situation where a file was being downloaded in the background and also being uploaded at the same time. I used memory buffers, and posted a gistwhich has working code. (It's GPL but that might not matter for you - in any case you can use the principles to do your own thing.)
更新:在回答这个问题时,我实现了类似的东西 - 在后台下载文件并同时上传文件的情况。我使用了内存缓冲区,并发布了一个包含工作代码的要点。(这是 GPL,但这对您来说可能无关紧要 - 在任何情况下,您都可以使用这些原则来做自己的事情。)
回答by EKS
The way i solved this is using the DirectoryWatcher/ FilesystemWatcher class, and when it triggers on the file you want you open a FileStream and read it to the end. And when im done reading i save the position of the reader, so next time the DirectoryWatcher/ FilesystemWatcher triggers i open a stream set the position to where i was last time.
我解决这个问题的方法是使用DirectoryWatcher/ FilesystemWatcher 类,当它触发文件时,您希望打开 FileStream 并将其读到最后。当我完成阅读时,我会保存阅读器的位置,因此下次DirectoryWatcher/ FilesystemWatcher 触发时,我会打开一个流,将位置设置为我上次所在的位置。
Calling FileStream.length is actualy very slow, i have had no performance issues with my solution ( I was im reading a "log" ranging from 10mb to 50 ish).
调用 FileStream.length 实际上非常慢,我的解决方案没有性能问题(我正在阅读从 10mb 到 50 ish 的“日志”)。
To me the solution i describe is very simple and easy to maintain, i would try it and profile it. I dont think your going to get any performance issues based on it. I do this when ppl are playing a multi threaded game, taking their entire CPU and nobody has complained that my parser is more demanding then the competing parsers.
对我来说,我描述的解决方案非常简单且易于维护,我会尝试并对其进行分析。我不认为你会基于它得到任何性能问题。我在 ppl 玩多线程游戏时这样做,占用了他们的整个 CPU,没有人抱怨我的解析器比竞争解析器要求更高。
回答by Jessitron
This worked with a StreamReader around a file, with the following steps:
这与围绕文件的 StreamReader 一起使用,步骤如下:
In the program that writes to the file, open it with read sharing, like this:
var out = new StreamWriter(File.Open("logFile.txt", FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read));
In the program that reads the file, open it with read-write sharing, like this:
using (FileStream fileStream = File.Open("logFile.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) using ( var file = new StreamReader(fileStream))
Before accessing the input stream, check whether the end has been reached, and if so, wait around a while.
while (file.EndOfStream) { Thread.Sleep(5); }
在写入文件的程序中,用读共享打开它,像这样:
var out = new StreamWriter(File.Open("logFile.txt", FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read));
在读取文件的程序中,以读写共享的方式打开,像这样:
using (FileStream fileStream = File.Open("logFile.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) using ( var file = new StreamReader(fileStream))
在访问输入流之前,检查是否已到达末尾,如果是,请稍等片刻。
while (file.EndOfStream) { Thread.Sleep(5); }
回答by Michael Mann
One other thing that might be useful is the FileStream class has a property on it called ReadTimeOut which is defined as:
另一件可能有用的事情是 FileStream 类有一个名为 ReadTimeOut 的属性,它定义为:
Gets or sets a value, in miliseconds, that determines how long the stream will attempt to read before timing out. (inherited from Stream)
获取或设置一个值(以毫秒为单位),该值确定流将在超时前尝试读取多长时间。(继承自 Stream)
This could be useful in that when your reads catch up to your writes the thread performing the reads may pause while the write buffer gets flushed. It would certianly be worth writing a small test to see if this property would help your cause in any way.
这可能很有用,因为当您的读取赶上您的写入时,执行读取的线程可能会在写入缓冲区被刷新时暂停。写一个小测试来看看这个属性是否会以任何方式帮助你的事业当然是值得的。
Are the read and write operations happening on the same object? If so you could write your own abstractions over the file and then write cross thread communication code such that the thread that is performing the writes and notify the thread performing the reads when it is done so that the thread doing the reads knows when to stop reading when it reaches EOF.
读写操作是否发生在同一个对象上?如果是这样,您可以在文件上编写自己的抽象,然后编写跨线程通信代码,以便执行写入的线程并在完成时通知执行读取的线程,以便执行读取的线程知道何时停止读取当它到达 EOF 时。