使用 node.js 下载图像

Downloading images with node.js

提问by Jonathan Ong

I'm trying to write a script to download images using node.js. This is what I have so far:

我正在尝试编写一个脚本来使用 node.js 下载图像。这是我到目前为止：

var maxLength = 10 // 10mb
var download = function(uri, callback) {
  http.request(uri)
    .on('response', function(res) {
      if (res.headers['content-length'] > maxLength*1024*1024) {
        callback(new Error('Image too large.'))
      } else if (!~[200, 304].indexOf(res.statusCode)) {
        callback(new Error('Received an invalid status code.'))
      } else if (!res.headers['content-type'].match(/image/)) {
        callback(new Error('Not an image.'))
      } else {
        var body = ''
        res.setEncoding('binary')
        res
          .on('error', function(err) {
            callback(err)
          })
          .on('data', function(chunk) {
            body += chunk
          })
          .on('end', function() {
            // What about Windows?!
            var path = '/tmp/' + Math.random().toString().split('.').pop()
            fs.writeFile(path, body, 'binary', function(err) {
              callback(err, path)
            })
          })
      }
    })
    .on('error', function(err) {
      callback(err)
    })
    .end();
}

I, however, want to make this more robust:

但是，我想让它更健壮：

1. Are there libraries that do this and do this better?
2. Is there a chance that response headers lie (about length, about content type)?
3. Are there any other status codes I should care about? Should I bother with redirects?
4. I think I read somewhere that binaryencoding is going to be deprecated. What do I do then?
5. How can I get this to work on windows?
6. Any other ways you can make this script better?

1. 是否有图书馆可以做到这一点并且做得更好？
2. 响应头是否有可能撒谎（关于长度，关于内容类型）？
3. 还有其他我应该关心的状态代码吗？我应该打扰重定向吗？
4. 我想我在某处读到binary编码将被弃用。那我该怎么办？
5. 我怎样才能让它在 Windows 上工作？
6. 还有其他方法可以使这个脚本更好吗？

Why: for a feature similar to imgur where users can give me a URL, I download that image, and rehost the image in multiple sizes.

为什么：对于类似于 imgur 的功能，用户可以给我一个 URL，我下载该图像，并以多种尺寸重新托管该图像。

回答by Cezary Wojtkowski

I'd suggest using the request module. Downloading a file is as simple as the following code:

我建议使用request 模块。下载文件就像下面的代码一样简单：

var fs = require('fs'),
    request = require('request');

var download = function(uri, filename, callback){
  request.head(uri, function(err, res, body){
    console.log('content-type:', res.headers['content-type']);
    console.log('content-length:', res.headers['content-length']);

    request(uri).pipe(fs.createWriteStream(filename)).on('close', callback);
  });
};

download('https://www.google.com/images/srpr/logo3w.png', 'google.png', function(){
  console.log('done');
});

回答by Nihey Takizawa

I ran into this problem some days ago, for a pure NodeJS answer I would suggest using Stream to merge the chunks together.

几天前我遇到了这个问题，对于纯 NodeJS 答案，我建议使用 Stream 将块合并在一起。

var http = require('http'),                                                
    Stream = require('stream').Transform,                                  
    fs = require('fs');                                                    

var url = 'http://www.google.com/images/srpr/logo11w.png';                    

http.request(url, function(response) {                                        
  var data = new Stream();                                                    

  response.on('data', function(chunk) {                                       
    data.push(chunk);                                                         
  });                                                                         

  response.on('end', function() {                                             
    fs.writeFileSync('image.png', data.read());                               
  });                                                                         
}).end();

The newest Node versions won't work well with binary strings, so merging chunks with strings is not a good idea when working with binary data.

最新的 Node 版本不能很好地处理二进制字符串，因此在处理二进制数据时将块与字符串合并不是一个好主意。

*Just be careful when using 'data.read()', it will empty the stream for the next 'read()' operation. If you want to use it more than once, store it somewhere.

*使用'data.read()'时要小心，它会为下一个'read()'操作清空流。如果您想多次使用它，请将其存放在某个地方。

回答by Grant Miller

You can use Axios(a promise-based HTTP client for Node.js) to download images in the order of your choosing in an asynchronous environment:

您可以使用Axios（基于promise的 Node.js 的 HTTP 客户端）在异步环境中按照您选择的顺序下载图像：

npm i axios

Then, you can use the following basic example to begin downloading images:

然后，您可以使用以下基本示例开始下载图像：

const fs = require('fs');
const axios = require('axios');

/* ============================================================
  Function: Download Image
============================================================ */

const download_image = (url, image_path) =>
  axios({
    url,
    responseType: 'stream',
  }).then(
    response =>
      new Promise((resolve, reject) => {
        response.data
          .pipe(fs.createWriteStream(image_path))
          .on('finish', () => resolve())
          .on('error', e => reject(e));
      }),
  );

/* ============================================================
  Download Images in Order
============================================================ */

(async () => {
  let example_image_1 = await download_image('https://example.com/test-1.png', 'example-1.png');

  console.log(example_image_1.status); // true
  console.log(example_image_1.error); // ''

  let example_image_2 = await download_image('https://example.com/does-not-exist.png', 'example-2.png');

  console.log(example_image_2.status); // false
  console.log(example_image_2.error); // 'Error: Request failed with status code 404'

  let example_image_3 = await download_image('https://example.com/test-3.png', 'example-3.png');

  console.log(example_image_3.status); // true
  console.log(example_image_3.error); // ''
})();

回答by Fareed Alnamrouti

if you want progress download try this:

如果你想要进度下载试试这个：

var fs = require('fs');
var request = require('request');
var progress = require('request-progress');

module.exports = function (uri, path, onProgress, onResponse, onError, onEnd) {
    progress(request(uri))
    .on('progress', onProgress)
    .on('response', onResponse)
    .on('error', onError)
    .on('end', onEnd)
    .pipe(fs.createWriteStream(path))
};

how to use:

如何使用：

  var download = require('../lib/download');
  download("https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png", "~/download/logo.png", function (state) {
            console.log("progress", state);
        }, function (response) {
            console.log("status code", response.statusCode);
        }, function (error) {
            console.log("error", error);
        }, function () {
            console.log("done");
        });

note: you should install both request & request-progress modules using:

注意：您应该使用以下方法安装请求和请求进度模块：

npm install request request-progress --save

回答by Chandan Chhajer

var fs = require('fs'),
http = require('http'),
https = require('https');

var Stream = require('stream').Transform;

var downloadImageToUrl = (url, filename, callback) => {

    var client = http;
    if (url.toString().indexOf("https") === 0){
      client = https;
     }

    client.request(url, function(response) {                                        
      var data = new Stream();                                                    

      response.on('data', function(chunk) {                                       
         data.push(chunk);                                                         
      });                                                                         

      response.on('end', function() {                                             
         fs.writeFileSync(filename, data.read());                               
      });                                                                         
   }).end();
};

downloadImageToUrl('https://www.google.com/images/srpr/logo11w.png', 'public/uploads/users/abc.jpg');

回答by Ahsan Ahmed

This is an extension to Cezary's answer. If you want to download it to a specific directory, use this. Also, use const instead of var. Its safe this way.

这是 Cezary 答案的扩展。如果要将其下载到特定目录，请使用它。另外，使用 const 而不是 var。这样就安全了。

const fs = require('fs');
const request = require('request');
var download = function(uri, filename, callback){
  request.head(uri, function(err, res, body){    
    request(uri).pipe(fs.createWriteStream(filename)).on('close', callback);
  });
};

download('https://www.google.com/images/srpr/logo3w.png', './images/google.png', function(){
  console.log('done');
});

回答by VladFr

Building on the above, if anyone needs to handle errors in the write/read streams, I used this version. Note the stream.read()in case of a write error, it's required so we can finish reading and trigger closeon the read stream.

在上述基础上，如果有人需要处理写入/读取流中的错误，我使用了这个版本。请注意，stream.read()在发生写入错误的情况下，这是必需的，因此我们可以完成读取并触发close读取流。

var download = function(uri, filename, callback){
  request.head(uri, function(err, res, body){
    if (err) callback(err, filename);
    else {
        var stream = request(uri);
        stream.pipe(
            fs.createWriteStream(filename)
                .on('error', function(err){
                    callback(error, filename);
                    stream.read();
                })
            )
        .on('close', function() {
            callback(null, filename);
        });
    }
  });
};