node.js 使用 gzip/deflate 压缩的简单 HTTP 请求

Easy HTTP requests with gzip/deflate compression

提问by wridgers

I'm trying to figure out how the best way to easily send HTTP/HTTPS requests and to handle gzip/deflate compressed responses along with cookies.

我试图找出轻松发送 HTTP/HTTPS 请求以及处理 gzip/deflate 压缩响应和 cookie 的最佳方式。

The best I found was https://github.com/mikeal/requestwhich handles everything exceptcompression. Is there a module or method that will do everything I ask?

我发现最好的是https://github.com/mikeal/request，它处理除压缩之外的所有内容。是否有一个模块或方法可以满足我的所有要求？

If not, can I combine request and zlib in some manner? I tried to combine zlib and http.ServerRequest, and it failed miserably.

如果没有，我可以以某种方式组合 request 和 zlib 吗？我试图结合 zlib 和http.ServerRequest，但它失败了。

回答by Ryan Knell

For anyone coming across this in recent times, the request library supports gzip decompression out of the box now. Use as follows:

对于最近遇到此问题的任何人，请求库现在支持开箱即用的 gzip 解压缩。使用方法如下：

request(
    { method: 'GET'
    , uri: 'http://www.google.com'
    , gzip: true
    }
  , function (error, response, body) {
      // body is the decompressed response body
      console.log('server encoded the data as: ' + (response.headers['content-encoding'] || 'identity'))
      console.log('the decoded data is: ' + body)
    }
  )

From the github readme https://github.com/request/request

来自 github 自述文件https://github.com/request/request

gzip - If true, add an Accept-Encoding header to request compressed content encodings from the server (if not already present) and decode supported content encodings in the response. Note: Automatic decoding of the response content is performed on the body data returned through request (both through the request stream and passed to the callback function) but is not performed on the response stream (available from the response event) which is the unmodified http.IncomingMessage object which may contain compressed data. See example below.

gzip - 如果为 true，则添加 Accept-Encoding 标头以从服务器请求压缩内容编码（如果尚未存在）并解码响应中支持的内容编码。注意：响应内容的自动解码是在请求返回的body数据上进行的（通过请求流和传递给回调函数），而不是在未修改的http响应流（从响应事件中获得）上进行.IncomingMessage 对象，其中可能包含压缩数据。请参阅下面的示例。

回答by jcreignou

Note: as of 2019, request has gzip decompression built in. You can still decompress requests manually using the below method.

注意：截至 2019 年，请求已内置 gzip 解压。您仍然可以使用以下方法手动解压请求。

You can simply combine requestand zlibwith streams.

您可以简单地将request和zlib与流结合起来。

Here is an example assuming you have a server listening on port 8000 :

这是一个示例，假设您有一个服务器侦听端口 8000 ：

var request = require('request'), zlib = require('zlib');

var headers = {
    'Accept-Encoding': 'gzip'
};

request({url:'http://localhost:8000/', 'headers': headers})
    .pipe(zlib.createGunzip()) // unzip
    .pipe(process.stdout); // do whatever you want with the stream

回答by user764155

Here's a working example that gunzips the response

这是一个对响应进行压缩的工作示例

function gunzipJSON(response){

    var gunzip = zlib.createGunzip();
    var json = "";

    gunzip.on('data', function(data){
        json += data.toString();
    });

    gunzip.on('end', function(){
        parseJSON(json);
    });

    response.pipe(gunzip);
}

Full code: https://gist.github.com/0xPr0xy/5002984

完整代码：https: //gist.github.com/0xPr0xy/5002984

回答by Dick Hardt

Check out the examples at http://nodejs.org/docs/v0.6.0/api/zlib.html#examples

在http://nodejs.org/docs/v0.6.0/api/zlib.html#examples查看示例

zlib is now built into node.

zlib 现在内置于 node.js 中。

回答by Yurik

Looking inside the source code- you must set the gzipparam on the request lib itself for gzip to work. Not sure if this was intentional or not, but this is the current implementation. No extra headers are needed.

查看源代码- 您必须gzip在请求库本身上设置参数才能使 gzip 工作。不确定这是否是故意的，但这是当前的实现。不需要额外的标题。

var request = require('request');
request.gzip = true;
request({url: 'https://...'},  // use encoding:null for buffer instead of UTF8
    function(error, response, body) { ... }
);

回答by Omar Mihilmy

All the answers here did not work and I was getting raw bytes back instead and the gzipflag was not working either. As it turns out you need to set the encoding to nullto prevent requests from transforming the response to utf-8 encoding and instead keeps the binary response.

这里的所有答案都不起作用，我取回了原始字节，而gzip标志也不起作用。事实证明，您需要将编码设置null为防止请求将响应转换为 utf-8 编码，而是保留二进制响应。

const request = require("request-promise-native");
const zlib = require("zlib");

const url = getURL("index.txt");
const dataByteBuffer = await request(url, { encoding: null });
const dataString = zlib.gunzipSync(response);