不使用 windows 文件缓存复制文件
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/48679/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Copy a file without using the windows file cache
提问by Ruvan Fernando
Anybody know of a way to copy a file from path A to path B and suppressing the Windows file system cache?
Typical use is copying a large file from a USB drive, or server to your local machine. Windows seems to swap everything out if the file is really big, e.g. 2GiB.
Prefer example in C#, but I'm guessing this would be a Win32 call of some sort if possible.
有人知道将文件从路径 A 复制到路径 B 并抑制 Windows 文件系统缓存的方法吗？
典型用途是将大文件从 USB 驱动器或服务器复制到本地计算机。如果文件真的很大，例如 2GiB，Windows 似乎会交换所有内容。更喜欢 C# 中的示例，但我猜如果可能的话，这将是某种 Win32 调用。
采纳答案by gabr
Even more important, there are FILE_FLAG_WRITE_THROUGH and FILE_FLAG_NO_BUFFERING.
更重要的是，还有 FILE_FLAG_WRITE_THROUGH 和 FILE_FLAG_NO_BUFFERING。
MSDN has a nice article on them both: http://support.microsoft.com/kb/99794
MSDN 上有一篇关于它们的好文章：http: //support.microsoft.com/kb/99794
回答by nietras
In C# I have found something like this to work, this can be changed to copy directly to destination file:
在 C# 中，我发现类似这样的东西可以工作，可以将其更改为直接复制到目标文件：
public static byte[] ReadAllBytesUnbuffered(string filePath)
{
const FileOptions FileFlagNoBuffering = (FileOptions)0x20000000;
var fileInfo = new FileInfo(filePath);
long fileLength = fileInfo.Length;
int bufferSize = (int)Math.Min(fileLength, int.MaxValue / 2);
bufferSize += ((bufferSize + 1023) & ~1023) - bufferSize;
using (var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.None,
bufferSize, FileFlagNoBuffering | FileOptions.SequentialScan))
{
long length = stream.Length;
if (length > 0x7fffffffL)
{
throw new IOException("File too long over 2GB");
}
int offset = 0;
int count = (int)length;
var buffer = new byte[count];
while (count > 0)
{
int bytesRead = stream.Read(buffer, offset, count);
if (bytesRead == 0)
{
throw new EndOfStreamException("Read beyond end of file EOF");
}
offset += bytesRead;
count -= bytesRead;
}
return buffer;
}
}
回答by Espo
I am not sure if this helps, but take a look at Increased Performance Using FILE_FLAG_SEQUENTIAL_SCAN.
我不确定这是否有帮助，但请查看使用 FILE_FLAG_SEQUENTIAL_SCAN 提高性能。
SUMMARY
There is a flag for CreateFile() called FILE_FLAG_SEQUENTIAL_SCAN which will direct the Cache Manager to access the file sequentially.
Anyone reading potentially large files with sequential access can specify this flag for increased performance. This flag is useful if you are reading files that are "mostly" sequential, but you occasionally skip over small ranges of bytes.
概括
CreateFile() 有一个名为 FILE_FLAG_SEQUENTIAL_SCAN 的标志，它将指示缓存管理器按顺序访问文件。
任何通过顺序访问读取潜在大文件的人都可以指定此标志以提高性能。如果您正在读取“大部分”顺序的文件，但偶尔会跳过小范围的字节，则此标志很有用。
回答by Gulzar Nazim
If you dont mind using a tool, ESEUTIL worked great for me.
如果您不介意使用工具，ESEUTIL 对我来说效果很好。
You can check out this blogentry comparing Buffered and NonBuffered IO functions and from where to get ESEUTIL.
您可以查看此博客条目，比较缓冲和非缓冲 IO 函数以及从何处获取 ESEUTIL。
copying some text from the technet blog:
从technet博客复制一些文本：
So looking at the definition of buffered I/O above, we can see where the perceived performance problems lie - in the file system cache overhead. Unbuffered I/O (or a raw file copy) is preferred when attempting to copy a large file from one location to another when we do not intend to access the source file after the copy is complete. This will avoid the file system cache overhead and prevent the file system cache from being effectively flushed by the large file data. Many applications accomplish this by calling CreateFile() to create an empty destination file, then using the ReadFile() and WriteFile() functions to transfer the data. CreateFile() - The CreateFile function creates or opens a file, file stream, directory, physical disk, volume, console buffer, tape drive, communications resource, mailslot, or named pipe. The function returns a handle that can be used to access an object. ReadFile() - The ReadFile function reads data from a file, and starts at the position that the file pointer indicates. You can use this function for both synchronous and asynchronous operations. WriteFile() - The WriteFile function writes data to a file at the position specified by the file pointer. This function is designed for both synchronous and asynchronous operation. For copying files around the network that are very large, my copy utility of choice is ESEUTIL which is one of the database utilities provided with Exchange.
因此，查看上面缓冲 I/O 的定义，我们可以看到感知到的性能问题所在——文件系统缓存开销。当我们不打算在复制完成后访问源文件时，尝试将大文件从一个位置复制到另一个位置时，首选无缓冲 I/O（或原始文件副本）。这将避免文件系统缓存开销并防止文件系统缓存被大文件数据有效刷新。许多应用程序通过调用 CreateFile() 创建一个空的目标文件，然后使用 ReadFile() 和 WriteFile() 函数来传输数据来实现这一点。CreateFile() - CreateFile 函数创建或打开文件、文件流、目录、物理磁盘、卷、控制台缓冲区、磁带驱动器、通信资源、邮槽或命名管道。该函数返回一个可用于访问对象的句柄。ReadFile() - ReadFile 函数从文件中读取数据，并从文件指针指示的位置开始。您可以将此函数用于同步和异步操作。WriteFile() - WriteFile 函数在文件指针指定的位置将数据写入文件。此功能是为同步和异步操作而设计的。为了在网络上复制非常大的文件，我选择的复制实用程序是 ESEUTIL，它是 Exchange 提供的数据库实用程序之一。WriteFile() - WriteFile 函数在文件指针指定的位置将数据写入文件。此功能是为同步和异步操作而设计的。为了在网络上复制非常大的文件，我选择的复制实用程序是 ESEUTIL，它是 Exchange 提供的数据库实用程序之一。WriteFile() - WriteFile 函数在文件指针指定的位置将数据写入文件。此功能是为同步和异步操作而设计的。为了在网络上复制非常大的文件，我选择的复制实用程序是 ESEUTIL，它是 Exchange 提供的数据库实用程序之一。
回答by Michael B?ckling
Eseutil is a correct answer, also since Win7 / 2008 R2, you can use the /j switch in Xcopy, which has the same effect.
Eseutil 是一个正确的答案，也是从 Win7 / 2008 R2 开始，您可以在 Xcopy 中使用 /j 开关，它具有相同的效果。
回答by Petros Matsakos
I understand this question was 11 years ago, nowadays there is robocopy which is kind of replacement for xcopy.
我知道这个问题是 11 年前的问题，现在有 robocopy，它是 xcopy 的替代品。
you need to check /J option
/J :: copy using unbuffered I/O (recommended for large files)
你需要检查 /J 选项
/J :: copy using unbuffered I/O (recommended for large files)
