node.js fs.createWriteStream 不会立即创建文件？

fs.createWriteStream does not immediately create file?

提问by Jang-hwan Kim

I have made a simple download from httpfunction as below (error handling is omitted for simplifcation):

我从 http函数做了一个简单的下载，如下所示（为了简化省略了错误处理）：

function download(url, tempFilepath, filepath, callback) {
    var tempFile = fs.createWriteStream(tempFilepath);
    http.request(url, function(res) {
        res.on('data', function(chunk) {
            tempFile.write(chunk);
        }).on('end', function() {
            tempFile.end();
            fs.renameSync(tempFile.path, filepath);
            return callback(filepath);
        })
    });
}

However, as I call download()tens of times asynchronously, it seldom reports error on fs.renameSynccomplaining it cannot find file at tempFile.path.

但是，当我download()异步调用数十次时，它很少报告错误，fs.renameSync抱怨它无法在tempFile.path.

Error: ENOENT, no such file or directory 'xxx'

I used the same list of urls to test it, and it failed about 30% of time. The same list of urls worked when downloaded one by one.

我使用相同的 url 列表来测试它，它失败了大约 30% 的时间。一一下载时，相同的网址列表有效。

Testing some more, I found out that the following code

测试了一些，我发现下面的代码

fs.createWriteStream('anypath');
console.log(fs.exist('anypath'));
console.log(fs.exist('anypath'));
console.log(fs.exist('anypath'));

does not always print true, but sometimes the first answer prints false.

不总是打印true，但有时打印第一个答案false。

I am suspecting that too many asynchronous fs.createWriteStreamcalls cannot guarantee the file creation. Is this true? Are there any methods to guarantee file creation?

我怀疑太多的异步fs.createWriteStream调用不能保证文件创建。这是真的？是否有任何方法可以保证文件创建？

回答by JohnnyHK

You shouldn't call writeon your tempFilewrite stream until you've received the 'open'event from the stream. The file won't exist until you see that event.

你不应该叫write上你tempFile，直到你已经收到了写流'open'从流事件。在您看到该事件之前，该文件将不存在。

For your function:

对于您的功能：

function download(url, tempFilepath, filepath, callback) {
    var tempFile = fs.createWriteStream(tempFilepath);
    tempFile.on('open', function(fd) {
        http.request(url, function(res) {
            res.on('data', function(chunk) {
                tempFile.write(chunk);
            }).on('end', function() {
                tempFile.end();
                fs.renameSync(tempFile.path, filepath);
                return callback(filepath);
            });
        });
    });
}

For your test:

对于您的测试：

var ws = fs.createWriteStream('anypath');
ws.on('open', function(fd) {
    console.log(fs.existsSync('anypath'));
    console.log(fs.existsSync('anypath'));
    console.log(fs.existsSync('anypath'));
});

回答by Dan Abramov

The accepted answer didn't download some of the last bytes for me.
Here's a Qversion that works correctly (but without the temp file).

接受的答案没有为我下载一些最后的字节。
这是一个可以正常工作的Q版本（但没有临时文件）。

'use strict';

var fs = require('fs'),
    http = require('http'),
    path = require('path'),
    Q = require('q');

function download(url, filepath) {
  var fileStream = fs.createWriteStream(filepath),
      deferred = Q.defer();

  fileStream.on('open', function () {
    http.get(url, function (res) {
      res.on('error', function (err) {
        deferred.reject(err);
      });

      res.pipe(fileStream);
    });
  }).on('error', function (err) {
    deferred.reject(err);
  }).on('finish', function () {
    deferred.resolve(filepath);
  });

  return deferred.promise;
}

module.exports = {
  'download': download
};

Note I'm listening to finishon file stream instead of endon response.

注意我正在听finish文件流而不是end响应。

回答by Doftom

Here is what I use to get it done:

这是我用来完成它的方法：

function download(url, dest) {
    return new Promise((resolve, reject) => {
        http.get(url, (res) => {
            if (res.statusCode !== 200) {
                var err = new Error('File couldn\'t be retrieved');
                err.status = res.statusCode;
                return reject(err);
            }
            var chunks = [];
            res.setEncoding('binary');
            res.on('data', (chunk) => {
                chunks += chunk;
            }).on('end', () => {
                var stream = fs.createWriteStream(dest);
                stream.write(chunks, 'binary');
                stream.on('finish', () => {
                    resolve('File Saved !');
                });
                res.pipe(stream);
            })
        }).on('error', (e) => {
            console.log("Error: " + e);
            reject(e.message);
        });
    })
};

回答by S.Mishra

I am working on uploading and downloading file (docx, pdf, text, etc) through nodejs request-promiseand requestlibraries.

我正在通过 nodejsrequest-promise和request库上传和下载文件（docx、pdf、文本等）。

Problem with request-promiseis that they don't promisify pipemethod from requestpackage. Hence, we need to do it in the old way.

问题request-promise在于他们没有pipe从request包中承诺方法。因此，我们需要用旧的方式来做。

I was able to come up with the hybrid solution, where I was able to use async/awaitand Promise()at same time. Here is the example:

我能想出的混合解决方案，在那里我能使用async/await，并Promise()在同一时间。这是示例：

    /**
     * Downloads the file.
     * @param {string} fileId : File id to be downloaded.
     * @param {string} downloadFileName : File name to be downloaded.
     * @param {string} downloadLocation : File location where it will be downloaded.
     * @param {number} version : [Optional] version of the file to be downloaded.
     * @returns {string}: Downloaded file's absolute path.
     */
    const getFile = async (fileId, downloadFileName, downloadLocation, version = undefined) => {
        try {
            const url = version ? `http://localhost:3000/files/${fileId}?version=${version}` : 
`${config.dms.url}/files/${fileUuid}`;
            const fileOutputPath = path.join(downloadLocation, fileName);

            const options = {
                method: 'GET',
                url: url,
                headers: {
                    'content-type': 'application/json',
                },
                resolveWithFullResponse: true
            }

            // Download the file and return the full downloaded file path.
            const downloadedFilePath = writeTheFileIntoDirectory(options, fileOutputPath);

            return downloadedFilePath;
        } catch (error) {
           console.log(error);
        }
    };

As you can see in above getFilemethod, we are using latest ES supported async/awaitfunctionality for asynchronous programming. Now, lets look into writeTheFileIntoDirectorymethod.

正如您在上面的getFile方法中看到的，我们正在使用最新的 ES 支持async/await功能进行异步编程。现在，让我们看看writeTheFileIntoDirectory方法。

/**
 * Makes REST API request and writes the file to the location provided.
 * @param {object} options : Request option to make REST API request.
 * @param {string} fileOutputPath : Downloaded file's absolute path.
 */
const writeTheFileIntoDirectory = (options, fileOutputPath) => {
    return new Promise((resolve, reject) => {
        // Get file downloaded.
        const stream = fs.createWriteStream(fileOutputPath);
        return request
            .get(options.url, options, (err, res, body) => {
                if (res.statusCode < 200 || res.statusCode >= 400) {
                    const bodyObj = JSON.parse(body);
                    const error = bodyObj.error;
                    error.statusCode = res.statusCode;
                    return reject(error);
                }
            })
            .on('error', error => reject(error))
            .pipe(stream)
            .on('close', () => resolve(fileOutputPath));
    });
}

The beauty of nodejs is that it support backward compatibility of different asynchronous implementation. If a method is returning promise, then awaitwill be kicked in and will wait for the method to be completed.

nodejs 的美妙之处在于它支持不同异步实现的向后兼容性。如果一个方法正在返回 promise，await则将被踢入并等待该方法完成。

Above writeTheFileIntoDirectorymethod will download the file and will return positively when the stream is closed successfully, else it will return error.

上述writeTheFileIntoDirectory方法将下载文件，并在流成功关闭时正返回，否则将返回错误。