Javascript node.js 中 fs.createReadStream 与 fs.readFile 的优缺点是什么？

What are the pros and cons of fs.createReadStream vs fs.readFile in node.js?

提问by Kent Brewster

I'm mucking about with node.js and have discovered two ways of reading a file and sending it down the wire, once I've established that it exists and have sent the proper MIME type with writeHead:

我正在使用 node.js 并发现了两种读取文件并将其发送到网络的方法，一旦我确定它存在并使用 writeHead 发送了正确的 MIME 类型：

// read the entire file into memory and then spit it out

fs.readFile(filename, function(err, data){
  if (err) throw err;
  response.write(data, 'utf8');
  response.end();
});

// read and pass the file as a stream of chunks

fs.createReadStream(filename, {
  'flags': 'r',
  'encoding': 'binary',
  'mode': 0666,
  'bufferSize': 4 * 1024
}).addListener( "data", function(chunk) {
  response.write(chunk, 'binary');
}).addListener( "close",function() {
  response.end();
});

Am I correct in assuming that fs.createReadStream might provide a better user experience if the file in question was something large, like a video? It feels like it might be less block-ish; is this true? Are there other pros, cons, caveats, or gotchas I need to know?

如果有问题的文件很大，比如视频，那么我假设 fs.createReadStream 可能提供更好的用户体验是否正确？感觉它可能不那么块状；这是真的？我还需要了解其他优点、缺点、警告或问题吗？

采纳答案by isaacs

A better approach, if you are just going to hook up "data" to "write()" and "close" to "end()":

更好的方法，如果您只是将“数据”连接到“write()”并将“关闭”连接到“end()”：

// 0.3.x style
fs.createReadStream(filename, {
  'bufferSize': 4 * 1024
}).pipe(response)

// 0.2.x style
sys.pump(fs.createReadStream(filename, {
  'bufferSize': 4 * 1024
}), response)

The read.pipe(write)or sys.pump(read, write)approach has the benefit of also adding flow control. So, if the write stream cannot accept data as quickly, it'll tell the read stream to back off, so as to minimize the amount of data getting buffered in memory.

该read.pipe(write)或sys.pump(read, write)方法具有还增加了流量控制的好处。因此，如果写入流不能尽快接受数据，它会告诉读取流退出，以尽量减少内存中缓冲的数据量。

The flags:"r"and mode:0666are implied by the fact that it is a FileReadStream. The binaryencoding is deprecated -- if an encoding is not specified, it'll just work with the raw data buffers.

在flags:"r"和mode:0666由事实，这是一个暗示FileReadStream。该binary编码已过时-如果没有指定编码，它会刚刚与原始数据缓冲区的工作。

Also, you could add some other goodies that will make your file serving a whole lot slicker:

此外，您可以添加一些其他的好东西，让您的文件服务更加流畅：

1. Sniff for req.headers.rangeand see if it matches a string like /bytes=([0-9]+)-([0-9]+)/. If so, you want to just stream from that start to end location. (Missing number means 0 or "the end".)
2. Hash the inode and creation time from the stat() call into an ETag header. If you get a request header with "if-none-match" matching that header, send back a 304 Not Modified.
3. Check the if-modified-sinceheader against the mtimedate on the stat object. 304 if it wasn't modified since the date provided.

1. 嗅探req.headers.range并查看它是否与像/bytes=([0-9]+)-([0-9]+)/. 如果是这样，您只想从该开始位置流式传输到结束位置。（缺少数字意味着 0 或“结束”。）
2. 将 stat() 调用中的 inode 和创建时间散列到 ETag 标头中。如果您收到与该标头匹配的“if-none-match”请求标头，请发回304 Not Modified.
3. 根据统计对象上的日期检查if-modified-since标题mtime。304 如果自提供的日期以来没有修改。

Also, in general, if you can, send a Content-Lengthheader. (You're stat-ing the file, so you should have this.)

此外，通常，如果可以，请发送Content-Length标头。（你正在stat-ing 文件，所以你应该有这个。）

回答by Christian Joudrey

fs.readFilewill load the entire file into memory as you pointed out, while as fs.createReadStreamwill read the file in chunks of the size you specify.

fs.readFile将按照您的指示将整个文件加载到内存中，而 asfs.createReadStream将以您指定大小的块读取文件。

The client will also start receiving data faster using fs.createReadStreamas it is sent out in chunks as it is being read, while as fs.readFilewill read the entire file out and only then start sending it to the client. This might be negligible, but can make a difference if the file is very big and the disks are slow.

客户端也将开始更快地接收数据fs.createReadStream，因为它在读取时以块的形式发送，而 asfs.readFile将读取整个文件，然后才开始将其发送到客户端。这可能可以忽略不计，但如果文件非常大且磁盘速度很慢，则可能会有所不同。

Think about this though, if you run these two functions on a 100MB file, the first one will use 100MB memory to load up the file while as the latter would only use at most 4KB.

不过想想看，如果你在一个 100MB 的文件上运行这两个函数，第一个将使用 100MB 内存来加载文件，而后者最多只能使用 4KB。

Edit: I really don't see any reason why you'd use fs.readFileespecially since you said you will be opening large files.

编辑：我真的不明白您为什么要使用任何理由，fs.readFile尤其是因为您说要打开大文件。

回答by Deen John

If it's a big file then "readFile" would hog the memory as it buffer all the file content in the memory and may hang your system. While ReadStream read in chunks.

如果它是一个大文件，那么“readFile”会占用内存，因为它会缓冲内存中的所有文件内容，并可能挂起您的系统。而 ReadStream 以块的形式读取。

Run this code and observe the memory usage in performance tab of task manager.

运行此代码并在任务管理器的性能选项卡中观察内存使用情况。

 var fs = require('fs');

const file = fs.createWriteStream('./big_file');


for(let i=0; i<= 1000000000; i++) {
  file.write('Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.\n');
}

file.end();


//..............
fs.readFile('./big_file', (err, data) => {
  if (err) throw err;
  console.log("done !!");
});

Infact,you won't see "done !!" message. "readFile" wouldn't be able to read the file content as buffer is not big enough to hold the file content.

事实上，你不会看到“完成！！” 信息。“readFile”将无法读取文件内容，因为缓冲区不足以容纳文件内容。

Now instead of "readFile", use readStream and monitor memory usage.

现在代替“readFile”，使用 readStream 并监视内存使用情况。

Note : code is taken from Samer buna Node course on Pluralsight

注意：代码取自 Pluralsight 上的 Samer buna Node 课程

回答by carl-johan.blomqvist

Another, perhaps not so well known thing, is that I believe that Node is better at cleaning up non-used memory after using fs.readFilecompared to fs.createReadStream. You should test this to verify what works best. Also, I know that by every new version of Node, this has gotten better (i.e. the garbage collector has become smarter with these types of situations).

另一个可能不太为人所知的事情是，我相信fs.readFile与fs.createReadStream. 您应该对此进行测试以验证什么最有效。另外，我知道每个新版本的 Node 都会变得更好（即垃圾收集器在这些类型的情况下变得更加智能）。