C# 何时使用内存映射文件？

When to use memory-mapped files?

提问by Pygmy

I have an application that receives chunks of data over the network, and writes these to disk. Once all chunks have been received, they can be decoded/recombined into the single file they actually represent.

我有一个应用程序，它通过网络接收数据块，并将这些数据写入磁盘。一旦接收到所有块，它们就可以被解码/重新组合成它们实际代表的单个文件。

I'm wondering if it's useful to use memory-mapped files or not - first for writing the single chunks to disk, second for the single file into which all of them are decoded.

我想知道使用内存映射文件是否有用 - 首先将单个块写入磁盘，其次用于将所有块解码成的单个文件。

My own feeling is that it might be useful for the second case only, anyone got some ideas on this?

我自己的感觉是它可能只对第二种情况有用，有人对此有一些想法吗？

Edit: It's a C# app, and I'm only planning an x64 version. (So running into the 'largest contigious free space' problem shouldn't be relevant)

编辑：这是一个 C# 应用程序，我只计划一个 x64 版本。（因此遇到“最大的连续可用空间”问题应该无关紧要）

采纳答案by user33675

Memory-mapped files are beneficial for scenarios where a relatively small portion (view) of a considerably larger file needs to be accessed repeatedly.

内存映射文件适用于需要重复访问相当大文件的相对较小部分（视图）的情况。

In this scenario, the operating system can help optimize the overall memory usage and paging behavior of the application by paging in and out only the most recently used portions of the mapped file.

在这种情况下，操作系统可以通过只调入和调出映射文件的最近使用的部分来帮助优化应用程序的整体内存使用和分页行为。

In addition, memory-mapped files can expose interesting features such as copy-on-write or serve as the basis of shared-memory.

此外，内存映射文件可以公开一些有趣的功能，例如写时复制或作为共享内存的基础。

For your scenario, memory-mapped files can help you assemble the file if the chunks arrive out of order. However, you would still need to know the final file size in advance.

对于您的方案，如果块无序到达，内存映射文件可以帮助您组装文件。但是，您仍然需要提前知道最终文件的大小。

Also, you should be accessing the files only once, for writing a chunk. Thus, a performance advantage over explicitly implemented asynchronous I/O is unlikely, but it may be easier and quicker to implement your file writer correctly.

此外，您应该只访问文件一次，以写入一个块。因此，与显式实现的异步 I/O 相比，性能优势不大可能，但正确实现文件编写器可能更容易和更快。

In .NET 4, Microsoft added support for memory-mapped files and there are some comprehensive articles with sample code, e.g. http://blogs.msdn.com/salvapatuel/archive/2009/06/08/working-with-memory-mapped-files-in-net-4.aspx.

在 .NET 4 中，Microsoft 添加了对内存映射文件的支持，并且有一些包含示例代码的综合文章，例如http://blogs.msdn.com/salvapatuel/archive/2009/06/08/working-with-memory-映射文件-in-net-4.aspx。

回答by Prof. Falken contract breached

I'd say both cases are relevant. Simply write the single chunks to their proper place in the memory mapped file, out of order, as they come in. This of course is only useful if you know where each chunk should go, like in a bittorrent downloader. If you have to perform some extra analysis to know where the chunk should go, the benefit of a memory mapped file might not be as large.

我想说这两种情况都是相关的。简单地将单个块写入内存映射文件中的适当位置，在它们进来时乱序。这当然只有在您知道每个块应该去哪里时才有用，就像在 bittorrent 下载器中一样。如果您必须执行一些额外的分析来知道块应该去哪里，那么内存映射文件的好处可能不会那么大。

回答by kervin

Memory-mapped files are primarily used for Inter-Process Communication or I/O performance improvement.

内存映射文件主要用于进程间通信或 I/O 性能改进。

In your case, are you trying to get better I/O performance?

就您而言，您是否想获得更好的 I/O 性能？

Hate to point out the obivious, but Wikipedia gives a good rundown of the situation... http://en.wikipedia.org/wiki/Memory-mapped_file

不想指出明显的问题，但维基百科对情况进行了很好的概述...... http://en.wikipedia.org/wiki/Memory-mapped_file

Specifically...

具体来说...

The memory mapped approach has its cost in minor page faults - when a block of data is loaded in page cache, but not yet mapped in to the process's virtual memory space. Depending on the circumstances, memory mapped file I/O can actually be substantially slower than standard file I/O.

内存映射方法在轻微页面错误方面有其代价 - 当一个数据块加载到页面缓存中，但尚未映射到进程的虚拟内存空间时。根据具体情况，内存映射文件 I/O 实际上可能比标准文件 I/O 慢得多。

It sounds like you're about to prematurely optimize for speed. Why not a regular file approach, and then refactor for MM files later if needed?

听起来您即将过早地优化速度。为什么不使用常规文件方法，然后在需要时为 MM 文件重构？