在 Node.js 中通过 HTTP 发送大图像数据
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/15001822/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Sending large image data over HTTP in Node.js
提问by Maroshii
In my development environment I have two servers. One sends and image to the other over a POSThttp request.
在我的开发环境中，我有两台服务器。一个通过POSThttp 请求将图像发送给另一个。
Client server does this:
客户端服务器这样做：
fs.readFile(rawFile.path,'binary',function (err, file){
restler.post("http://0.0.0.0:5000",{
data: file,
headers:{
"Content-Type": rawFile.type,
}
}).on('complete',function(data,response){
console.log(data);
res.send("file went through")
})
The server that recieves the request does this:
接收请求的服务器执行以下操作：
server.post('/',function(req,res,next){
fs.writeFileSync("test.png",req.body,"binary",function(err){
if(err) throw err;
res.send("OK")
})
})
If i send a small image it works fine. However, if i send a large image although the file is saved correctly only the first upper portion of the image is displayed. The rest is black. Image size is correct.
如果我发送一个小图像它工作正常。但是，如果我发送大图像，尽管文件已正确保存，但仅显示图像的第一部分。其余为黑色。图像大小正确。
I guess it's just the first chunk of the image that's being written on the file.
I've tried creating a readStreamand a writeStreambut it doesn't seem to work:
我想这只是写入文件的图像的第一块。我试过创建 areadStream和 awriteStream但它似乎不起作用：
req.body.pipe(fs.createWriteStream('test.png'))
Can i stream directly from the binary data and pipeit into the file? For what i've seen, readStreamis often used to stream from files not raw binary data.
我可以直接从二进制数据流式传输pipe到文件中吗？对于我所看到的，readStream通常用于从文件而不是原始二进制数据进行流式传输。
I read a few posts but it doesn't seem to work for me.
我阅读了一些帖子，但它似乎对我不起作用。
I'm using restlermodule in the client server and restifyin the other.
我restler在客户端服务器和restify其他服务器中使用模块。
Thanks!
谢谢！
回答by josh3736
Sorry to be blunt, but there's a lot wrong here.
恕我直言，但这里有很多错误。
readFilereads the entire contentsof a file into memory before invoking the callback, at which point you begin uploading the file.
readFile在调用回调之前将文件的全部内容读入内存，此时您开始上传文件。
This is bad–especially when dealing with large files like images–because there's really no reason to read the file into memory. It's wasteful; and under load, you'll find that your server will run out of memory and crash.
这很糟糕——尤其是在处理像图像这样的大文件时——因为真的没有理由将文件读入内存。很浪费；在负载下，您会发现您的服务器将耗尽内存并崩溃。
Instead, you want to get a stream, which emits chunks of data as they're read from disk. All you have to do is pass those chunks along to your upload stream (pipe), and then discard the data from memory. In this way, you never use more than a small amount of buffer memory.
相反，您想要获取一个stream，它在从磁盘读取数据时发出数据块。您所要做的就是将这些块传递到您的上传流 ( pipe)，然后从内存中丢弃数据。这样，您永远不会使用超过少量的缓冲存储器。
(A readable stream's default behavior is to deal in raw binary data; it's only if you pass an encodingthat it deals in text.)
（可读流的默认行为是处理原始二进制数据；只有当您传递它时encoding，它才处理文本。）
The requestmodule makes this especially easy:
该请求模块使得该特别容易：
fs.createReadStream('test.png').pipe(request.post('http://0.0.0.0:5000/'));
On the server, you have a larger problem. Never use *Syncmethods.It blocks your server from doing anything(like responding to other requests) until the entire file is flushed to disk, which can take seconds.
在服务器上，您遇到了更大的问题。 永远不要使用 *Sync方法。它会阻止您的服务器执行任何操作（例如响应其他请求），直到整个文件刷新到磁盘，这可能需要几秒钟。
So instead, we want to take the incoming data stream and pipe it to a filesystem stream. You were on the right track originally; the reason that req.body.pipe(fs.createWriteStream('test.png'))didn't work is because bodyis not a stream.
因此，我们希望获取传入的数据流并将其通过管道传输到文件系统流。你原本是在正确的轨道上；req.body.pipe(fs.createWriteStream('test.png'))不起作用的原因是因为body不是流。
bodyis generated by the bodyParsermiddleware. In restify, that middleware acts much like readFilein that it buffers the entire incoming request-entity in memory. In this case, we don't want that. Disable the body parser middleware.
body由bodyParser中间件生成。在 restify 中，该中间件的作用很像readFile它在内存中缓冲整个传入的请求实体。在这种情况下，我们不希望那样。禁用正文解析器中间件。
So where is the incoming data stream? It is the reqobject itself. restify's Requestinherits node's http.IncomingMessage, which is a readable stream. So:
那么传入的数据流在哪里？它是req对象本身。restifyRequest继承了 node 的http.IncomingMessage，这是一个可读流。所以：
fs.createWriteStream('test.png').pipe(req);
I should also mention that this all works so simply because there's no form parsing overhead involved. request simply sends the file with no multipart/form-datawrappers:
我还应该提到这一切都如此简单，因为没有涉及表单解析开销。request 只是发送没有multipart/form-data包装器的文件：
POST / HTTP/1.1
host: localhost:5000
content-type: application/octet-stream
Connection: keep-alive
Transfer-Encoding: chunked
<image data>...
This means that a browser could not post a file to this URL. If that's a need, look in to formidable, which does streaming parsing of request-entities.
这意味着浏览器无法将文件发布到此 URL。如果需要，请查看formidable，它对请求实体进行流式解析。
回答by Jean-Philippe Leclerc
I dont know much about restler. But posting an image is a multipart request.
我不太了解restler。但是发布图像是一个多部分请求。
restler.post("http://0.0.0.0:5000",{
data: restler.file(path, filename, fileSize, encoding, contentType),
multipart: true
})
回答by John Josef
I tried the solution above, and if you've just moving uploaded files or something the following works much better:
我尝试了上面的解决方案，如果您只是移动上传的文件或以下内容，效果会更好：
fs.rename(path, newPath, callback(err) {});
fs.rename(path, newPath, callback(err) {});
I was uploading files over 200MB and would encounter errors using streams, sync or async.
我上传的文件超过 200MB，在使用流、同步或异步时会遇到错误。
