使用请求进行 nodejs 编码

nodejs encoding using request

提问by hippie

I am trying to get the correct encoding with request.

我正在尝试通过请求获得正确的编码。

request.get({
    "uri":'http://www.bold.dk/tv/',
    "encoding": "text/html;charset='charset=utf-8'"
  },
  function(err, resp, body){    
    console.log(body);
  }
);

No matter what I do the encoding of the danish chars are not right.

无论我做什么，丹麦字符的编码都不正确。

Any thoughts?

有什么想法吗？

采纳答案by woolfi makkinan

Maybe your trouble is in 'Accept-Encoding'header. Let's say you have Headers like 'Accept-Encoding': 'gzip,deflate'

也许你的麻烦出在'Accept-Encoding'标题上。假设你有这样的标题'Accept-Encoding': 'gzip,deflate'

If it's so, you have 2 ways to fixing this:

如果是这样，您有两种方法可以解决此问题：

1. Remove this Header

2. Use the following code to unzip the data:

   const req = request(options, res => {
       let buffers = []
       let bufferLength = 0
       let strings = []
   
       const getData = chunk => {
           if (!Buffer.isBuffer(chunk)) {
               strings.push(chunk)
           } else if (chunk.length) {
               bufferLength += chunk.length
               buffers.push(chunk)
           }
       }
   
       const endData = () => {
           let response = {code: 200, body: ''}
           if (bufferLength) {
               response.body = Buffer.concat(buffers, bufferLength)
               if (options.encoding !== null) {
                   response.body = response.body.toString(options.encoding)
               }
               buffers = []
               bufferLength = 0
           } else if (strings.length) {
               if (options.encoding === 'utf8' && strings[0].length > 0 && strings[0][0] === '\uFEFF') {
                   strings[0] = strings[0].substring(1)
               }
               response.body = strings.join('')
           }
           console.log('response', response)
       };
   
       switch (res.headers['content-encoding']) {
           // or, just use zlib.createUnzip() to handle both cases
           case 'gzip':
               res.pipe(zlib.createGunzip())
                   .on('data', getData)
                   .on('end', endData)
               break;
           case 'deflate':
               res.pipe(zlib.createInflate())
                   .on('data', getData)
                   .on('end', endData)
               break;
           default:
               res.pipe(zlib.createInflate())
                   .on('data', getData)
                   .on('end', endData)
               break;
       }
   });
   

1. 删除此标题

2. 使用以下代码解压数据：

   const req = request(options, res => {
       let buffers = []
       let bufferLength = 0
       let strings = []
   
       const getData = chunk => {
           if (!Buffer.isBuffer(chunk)) {
               strings.push(chunk)
           } else if (chunk.length) {
               bufferLength += chunk.length
               buffers.push(chunk)
           }
       }
   
       const endData = () => {
           let response = {code: 200, body: ''}
           if (bufferLength) {
               response.body = Buffer.concat(buffers, bufferLength)
               if (options.encoding !== null) {
                   response.body = response.body.toString(options.encoding)
               }
               buffers = []
               bufferLength = 0
           } else if (strings.length) {
               if (options.encoding === 'utf8' && strings[0].length > 0 && strings[0][0] === '\uFEFF') {
                   strings[0] = strings[0].substring(1)
               }
               response.body = strings.join('')
           }
           console.log('response', response)
       };
   
       switch (res.headers['content-encoding']) {
           // or, just use zlib.createUnzip() to handle both cases
           case 'gzip':
               res.pipe(zlib.createGunzip())
                   .on('data', getData)
                   .on('end', endData)
               break;
           case 'deflate':
               res.pipe(zlib.createInflate())
                   .on('data', getData)
                   .on('end', endData)
               break;
           default:
               res.pipe(zlib.createInflate())
                   .on('data', getData)
                   .on('end', endData)
               break;
       }
   });
   

回答by Jens Mikkelsen

You can use iconv (lite) to convert this. You also need to tell request not to actively set the encoding to the default of UTF-8 by setting the encoding property to null. Therefore you should do:

您可以使用 iconv (lite) 来转换它。您还需要通过将 encoding 属性设置为 null 来告诉 request 不要主动将编码设置为默认的 UTF-8。因此你应该这样做：

var iconv = require('iconv-lite');
request.get({
    uri:'http://www.bold.dk/tv/',
    encoding: null
  },
  function(err, resp, body){    
    var bodyWithCorrectEncoding = iconv.decode(body, 'iso-8859-1');
    console.log(bodyWithCorrectEncoding);
  }
);

回答by Chengyzh

I have the same problem, with request v2.88.0.

我有同样的问题，与request v2.88.0.

Refer to woolfi makkinan's answer, I got a simple way to solve the problem.

参考woolfi makkinan 的回答，我找到了解决问题的简单方法。

request.get({
    "uri": 'http://www.bold.dk/tv/',
    "encoding": "text/html;charset='charset=utf-8'",
    "gzip": true // notice this config
  },
  function(err, resp, body){    
    console.log(body);
  }
);

Add gzip: trueto requestoptions, requestwill deal with gzip, and then blob can convert to string correctly. ?

添加gzip: true到request选项，request将处理gzip，然后blob可以正确转换为字符串。?