Javascript 使用 mongoose 在 MongoDB 中批量插入
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/37379180/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Bulk insert in MongoDB using mongoose
提问by javascript novice
I currently have a collection in Mongodb say "Collection1". I have the following array of objects that need to be into inserted into MongoDB. I am using Mongoose API. For now, I am iterating through the array and inserting each of them into mongo. This is ok for now, but will be a problem when the data is too big. I need a way of inserting the data in bulk into MongoDB without repetition. I am not sure how to do this. I could not find a bulk option in Mongoose.
我目前在 Mongodb 中有一个集合，比如“Collection1”。我有以下需要插入到 MongoDB 中的对象数组。我正在使用猫鼬 API。现在，我正在遍历数组并将它们中的每一个插入到 mongo 中。这暂时没问题，但是当数据太大时会出现问题。我需要一种将数据批量插入 MongoDB 而不重复的方法。我不知道该怎么做。我在猫鼬中找不到批量选项。
My code below
我的代码如下
myData = [Obj1,Obj2,Obj3.......]
myData.forEach(function(ele){
//console.log(ele)
saveToMongo(ele);
});
function saveToMongo(obj){
(new Collection1(obj)).save(function (err, response) {
if (err) {
// console.log('Error while inserting: ' + obj.name + " " +err);
} else {
// console.log('Data successfully inserted');
}
});
return Collection1(obj);
}
回答by chridam
You might want to use the insertMany()method here if you're using the latest Mongoose version 4.4.Xand greater, which essentially uses Model.collection.insertMany()under the hood and the driver might handle parallelizing >= 1000docs for you.
insertMany()如果您使用的是最新的 Mongoose 版本4.4.X及更高版本，您可能希望在此处使用该方法，它本质上是在幕后使用Model.collection.insertMany()，驱动程序可能会>= 1000为您处理并行化文档。
myData = [Obj1, Obj2, Obj3.......];
Collection1.insertMany(myData, function(error, docs) {});
or using Promises for better error handling
或者使用 Promises 来更好地处理错误
Collection1.insertMany(myData)
.then(function(docs) {
// do something with docs
})
.catch(function(err) {
// error handling here
});
It works by creating a bunch of documents, calls .validate()on them in parallel, and then calls the underlying driver's insertMany()on the result of toObject({ virtuals: false });of each doc.
Although insertMany()doesn't trigger pre-save hooks, it has better performance because it only makes 1 round-trip to the server rather than 1 for each document.
它的工作原理是创建一堆文档，.validate()并行调用它们，然后根据每个文档insertMany()的结果调用底层驱动程序toObject({ virtuals: false });。虽然insertMany()不触发预保存钩子，但它具有更好的性能，因为它只对服务器进行 1 次往返，而不是对每个文档进行 1 次往返。
For Mongoose versions ~3.8.8, ~3.8.22, 4.xwhich support MongoDB Server >=2.6.x, you could use the Bulk APIas follows
对于~3.8.8, ~3.8.22, 4.x支持 MongoDB Server 的Mongoose 版本>=2.6.x，您可以使用Bulk API如下
var bulk = Collection1.collection.initializeOrderedBulkOp(),
counter = 0;
myData.forEach(function(doc) {
bulk.insert(doc);
counter++;
if (counter % 500 == 0) {
bulk.execute(function(err, r) {
// do something with the result
bulk = Collection1.collection.initializeOrderedBulkOp();
counter = 0;
});
}
});
// Catch any docs in the queue under or over the 500's
if (counter > 0) {
bulk.execute(function(err,result) {
// do something with the result here
});
}
回答by user3227295
you can pass an array of objects to mongoose model create function
您可以将一组对象传递给猫鼬模型创建函数
var Collection1 = mongoose.model('Collection1');
Collection1.create(myData,function(err){
if(err) ...
});
