Javascript 使用 Node.js 将多条记录插入 Mongodb 的正确方法
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/34530348/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Correct way to insert many records into Mongodb with Node.js
提问by Richard Hensman
I was wondering what is the correct way to do bulk inserts into Mongodb (although could be any other database) with Node.js
我想知道使用 Node.js 批量插入 Mongodb（虽然可以是任何其他数据库）的正确方法是什么
I have written the following code as an example, although I believe it is floored as db.close() may be run before all the asynchronous collection.insert calls have completed.
我已经编写了以下代码作为示例，尽管我相信它是因为 db.close() 可能会在所有异步 collection.insert 调用完成之前运行。
MongoClient.connect('mongodb://127.0.0.1:27017/test', function (err, db) {
var i, collection;
if (err) {
throw err;
}
collection = db.collection('entries');
for (i = 0; i < entries.length; i++) {
collection.insert(entries[i].entry);
}
db.close();
});
回答by chridam
If your MongoDB server is 2.6 or newer, it would be better to take advantage of using a write commands Bulk APIthat allow for the execution of bulk insert operations which are simply abstractions on top of the server to make it easy to build bulk operations and thus get perfomance gains with your update over large collections.
如果您的 MongoDB 服务器是 2.6 或更高版本，最好利用写入命令Bulk API，它允许执行批量插入操作，这些操作只是服务器顶部的抽象，以便轻松构建批量操作和从而通过对大型集合的更新获得性能提升。
Sending the bulk insert operations in batches results in less traffic to the server and thus performs efficient wire transactions by not sending everything all in individual statements, but rather breaking up into manageable chunks for server commitment. There is also less time waiting for the response in the callback with this approach.
批量发送批量插入操作会减少到服务器的流量，从而通过不在单独的语句中发送所有内容，而是分解为服务器承诺的可管理块来执行高效的线事务。使用这种方法在回调中等待响应的时间也更少。
These bulk operations come mainly in two flavours:
这些批量操作主要有两种形式：
- Ordered bulk operations. These operations execute all the operation in order and error out on the first write error.
- Unordered bulk operations. These operations execute all the operations in parallel and aggregates up all the errors. Unordered bulk operations do not guarantee order of execution.
- 订购批量操作。这些操作按顺序执行所有操作，并在第一个写入错误时出错。
- 无序批量操作。这些操作并行执行所有操作并汇总所有错误。无序批量操作不保证执行顺序。
Note, for older servers than 2.6 the API will downconvert the operations. However it's not possible to downconvert 100% so there might be some edge cases where it cannot correctly report the right numbers.
请注意，对于比 2.6 更旧的服务器，API 将向下转换操作。但是，不可能 100% 下变频，因此可能存在一些无法正确报告正确数字的边缘情况。
In your case, you could implement the Bulk API insert operation in batches of 1000 like this:
在您的情况下，您可以像这样以 1000 个批次实现批量 API 插入操作：
For MongoDB 3.2+using bulkWrite
对于 MongoDB 3.2+使用bulkWrite
var MongoClient = require('mongodb').MongoClient;
var url = 'mongodb://localhost:27017/test';
var entries = [ ... ] // a huge array containing the entry objects
var createNewEntries = function(db, entries, callback) {
// Get the collection and bulk api artefacts
var collection = db.collection('entries'),
bulkUpdateOps = [];
entries.forEach(function(doc) {
bulkUpdateOps.push({ "insertOne": { "document": doc } });
if (bulkUpdateOps.length === 1000) {
collection.bulkWrite(bulkUpdateOps).then(function(r) {
// do something with result
});
bulkUpdateOps = [];
}
})
if (bulkUpdateOps.length > 0) {
collection.bulkWrite(bulkUpdateOps).then(function(r) {
// do something with result
});
}
};
For MongoDB <3.2
对于 MongoDB <3.2
var MongoClient = require('mongodb').MongoClient;
var url = 'mongodb://localhost:27017/test';
var entries = [ ... ] // a huge array containing the entry objects
var createNewEntries = function(db, entries, callback) {
// Get the collection and bulk api artefacts
var collection = db.collection('entries'),
bulk = collection.initializeOrderedBulkOp(), // Initialize the Ordered Batch
counter = 0;
// Execute the forEach method, triggers for each entry in the array
entries.forEach(function(obj) {
bulk.insert(obj);
counter++;
if (counter % 1000 == 0 ) {
// Execute the operation
bulk.execute(function(err, result) {
// re-initialise batch operation
bulk = collection.initializeOrderedBulkOp();
callback();
});
}
});
if (counter % 1000 != 0 ){
bulk.execute(function(err, result) {
// do something with result
callback();
});
}
};
Call the createNewEntries()function.
调用createNewEntries()函数。
MongoClient.connect(url, function(err, db) {
createNewEntries(db, entries, function() {
db.close();
});
});
回答by Arjan Frans
回答by Mrityunjay pandey
var MongoClient = require('mongodb').MongoClient;
var url = 'mongodb://localhost:27017/test';
var data1={
name:'Data1',
work:'student',
No:4355453,
Date_of_birth:new Date(1996,10,17)
};
var data2={
name:'Data2',
work:'student',
No:4355453,
Date_of_birth:new Date(1996,10,17)
};
MongoClient.connect(url, function(err, db) {
if(err!=null){
return console.log(err.message)
}
//insertOne
db.collection("App").insertOne(data1,function (err,data) {
if(err!=null){
return console.log(err);
}
console.log(data.ops[0]);
});
//insertMany
var Data=[data1,data2];
db.collection("App").insertMany(Data,forceServerObjectId=true,function (err,data) {
if(err!=null){
return console.log(err);
}
console.log(data.ops);
});
db.close();
});
回答by Dimuthu
New in version 3.2.
3.2 版中的新功能。
The db.collection.bulkWrite()method provides the ability to perform bulk insert, update, and remove operations. MongoDB also supports bulk insert through the db.collection.insertMany().
所述db.collection.bulkWrite（）方法提供了执行批量插入，更新和删除操作的能力。MongoDB 还支持通过db.collection.insertMany() 进行批量插入。
In bulkWrite it is supporting only insertOne, updateOne, updateMany, replaceOne, deleteOne, deleteMany
在bulkWrite中它只支持insertOne、updateOne、updateMany、replaceOne、deleteOne、deleteMany
In your case to insert data using single line of code, it can use insertMany option.
在您使用单行代码插入数据的情况下，它可以使用 insertMany 选项。
MongoClient.connect('mongodb://127.0.0.1:27017/test', function (err, db) {
var i, collection;
if (err) {
throw err;
}
collection = db.collection('entries');
collection.insertMany(entries)
db.close();
});