库推荐：NodeJs 读取 csv 文件
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/23080413/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Library Recommendations: NodeJs reading csv file
提问by lonelymo
With nodejs I want to parse a .csv file of 10000 records and do some operation on each row. I tried using http://www.adaltas.com/projects/node-csv. I couldnt get this to pause at each row. This just reads through all the 10000 records. I need to do the following
使用 nodejs，我想解析一个包含 10000 条记录的 .csv 文件并对每一行进行一些操作。我尝试使用http://www.adaltas.com/projects/node-csv。我无法让它在每一行暂停。这只是读取所有 10000 条记录。我需要做以下事情
- read csv line by line
- perform time consuming operation on each line
- go to the next line
- 逐行读取csv
- 对每一行进行耗时的操作
- 转到下一行
Can anyone please suggest any alternative ideas here?
任何人都可以在这里提出任何替代想法吗？
采纳答案by Risto Novik
Seems like you need to use some stream based solution, there existed already such libraries so before reinventing yourself, try this library, which also includes validation support. https://www.npmjs.org/package/fast-csv
似乎您需要使用一些基于流的解决方案，已经存在这样的库，所以在重塑自己之前，请尝试这个库，它也包括验证支持。https://www.npmjs.org/package/fast-csv
回答by prule
My current solution uses the async module to execute in series:
我当前的解决方案使用 async 模块串行执行：
var fs = require('fs');
var parse = require('csv-parse');
var async = require('async');
var inputFile='myfile.csv';
var parser = parse({delimiter: ','}, function (err, data) {
async.eachSeries(data, function (line, callback) {
// do something with the line
doSomething(line).then(function() {
// when processing finishes invoke the callback to move to the next one
callback();
});
})
});
fs.createReadStream(inputFile).pipe(parser);
回答by vineet
I used this way:-
我用这种方式：-
var fs = require('fs');
var parse = require('csv-parse');
var csvData=[];
fs.createReadStream(req.file.path)
.pipe(parse({delimiter: ':'}))
.on('data', function(csvrow) {
console.log(csvrow);
//do something with csvrow
csvData.push(csvrow);
})
.on('end',function() {
//do something with csvData
console.log(csvData);
});
回答by Pransh Tiwari
- This solution uses
csv-parserinstead ofcsv-parseused in some of the answers above. csv-parsercame around 2 years aftercsv-parse.- Both of them solve the same purpose, but personally I have found
csv-parserbetter, as it is easy to handle headers through it.
- 此解决方案使用
csv-parser而不是csv-parse在上述某些答案中使用。 csv-parser大约在 2 年后出现csv-parse。- 它们都解决了相同的目的，但我个人发现
csv-parser更好，因为通过它很容易处理标题。
Install the csv-parser first:
首先安装 csv-parser：
npm install csv-parser
So suppose you have a csv-file like this:
所以假设你有一个像这样的 csv 文件：
NAME, AGE
Lionel Messi, 31
Andres Iniesta, 34
You can perform the required operation as:
您可以执行所需的操作：
const fs = require('fs');
const csv = require('csv-parser');
fs.createReadStream(inputFilePath)
.pipe(csv())
.on('data', function(data){
try {
console.log("Name is: "+data.NAME);
console.log("Age is: "+data.AGE);
//perform the operation
}
catch(err) {
//error handler
}
})
.on('end',function(){
//some final operation
});
For further reading refer
如需进一步阅读，请参阅
回答by adnan kamili
In order to pause the streaming in fast-csvyou can do the following:
为了在fast-csv 中暂停流式传输，您可以执行以下操作：
let csvstream = csv.fromPath(filePath, { headers: true })
.on("data", function (row) {
csvstream.pause();
// do some heavy work
// when done resume the stream
csvstream.resume();
})
.on("end", function () {
console.log("We are done!")
})
.on("error", function (error) {
console.log(error)
});
回答by krwck
The node-csv project that you are referencing is completely sufficient for the task of transforming each row of a large portion of CSV data, from the docs at: http://csv.adaltas.com/transform/:
您正在引用的 node-csv 项目对于转换大部分 CSV 数据的每一行的任务完全足够，来自以下文档：http: //csv.adaltas.com/transform/：
csv()
.from('82,Preisner,Zbigniew\n94,Gainsbourg,Serge')
.to(console.log)
.transform(function(row, index, callback){
process.nextTick(function(){
callback(null, row.reverse());
});
});
From my experience, I can say that it is also a rather fast implementation, I have been working with it on data sets with near 10k records and the processing times were at a reasonable tens-of-milliseconds level for the whole set.
根据我的经验，我可以说它也是一个相当快的实现，我一直在处理接近 10k 记录的数据集，并且整个集的处理时间在合理的几十毫秒级别。
Rearding jurka's stream based solution suggestion: node-csv IS stream based and follows the Node.js' streaming API.
Rearding jurka的基于流的解决方案建议：node-csv 基于流并遵循 Node.js 的流 API。
回答by ramachandrareddy reddam
The Fast-CSVnpm module can read data line-by-line from csv file.
该快速CSVNPM模块可以读取数据线，由线从csv文件。
Here is an example:
下面是一个例子：
let csv= require('fast-csv');
var stream = fs.createReadStream("my.csv");
csv
.fromStream(stream, {headers : true})
.on("data", function(data){
console.log('I am one line of data', data);
})
.on("end", function(){
console.log("done");
});
回答by alexkb
I needed an async csv reader and originally tried @Pransh Tiwari's answer but couldn't get it working with awaitand util.promisify(). Eventually I came across node-csvtojson, which pretty much does the same as csv-parser, but with promises. Here is an example usage of csvtojson in action:
我需要一个异步 csv 阅读器，最初尝试@Pransh Tiwari 的答案，但无法使用await和util.promisify()。最终我遇到了node-csvtojson，它与 csv-parser 几乎一样，但有承诺。以下是 csvtojson 的使用示例：
const csvToJson = require('csvtojson');
const processRecipients = async () => {
const recipients = await csvToJson({
trim:true
}).fromFile('./recipients.csv');
// Code executes after recipients are fully loaded.
recipients.forEach((recipient) => {
console.log(recipient.name, recipient.email);
});
};
回答by HMagdy
Workaround for doing this task with await/async:
使用await/async执行此任务的解决方法：
const csv = require('csvtojson')
const csvFilePath = 'data.csv'
const array = await csv().fromFile(csvFilePath);
回答by Andrea Perdicchia
this is my solution to get csv file from external url
这是我从外部 url 获取 csv 文件的解决方案
const parse = require( 'csv-parse/lib/sync' );
const axios = require( 'axios' );
const readCSV = ( module.exports.readCSV = async ( path ) => {
try {
const res = await axios( { url: path, method: 'GET', responseType: 'blob' } );
let records = parse( res.data, {
columns: true,
skip_empty_lines: true
} );
return records;
} catch ( e ) {
console.log( 'err' );
}
} );
readCSV('https://urltofilecsv');
