node.js 猫鼬（mongodb）批量插入？

Mongoose (mongodb) batch insert?

提问by Geuis

Does Mongoose v3.6+support batch inserts now? I've searched for a few minutes but anything matching this query is a couple of years old and the answer was an unequivocal no.

的确猫鼬V3.6 +现在支持批量插入？我已经搜索了几分钟，但与此查询匹配的任何内容都是几年前的，答案是明确的“否”。

Edit:

编辑：

For future reference, the answer is to use Model.create(). create()accepts an array as its first argument, so you can pass your documents to be inserted as an array.

为了将来参考，答案是使用Model.create(). create()接受一个数组作为它的第一个参数，因此您可以传递要作为数组插入的文档。

See Model.create() documentation

请参阅Model.create() 文档

回答by Lucio Paiva

Model.create() vs Model.collection.insert(): a faster approach

Model.create() 与 Model.collection.insert()：更快的方法

Model.create()is a bad way to do inserts if you are dealing with a very large bulk. It will be very slow. In that case you should use Model.collection.insert, which performs much better. Depending on the size of the bulk, Model.create()will even crash! Tried with a million documents, no luck. Using Model.collection.insertit took just a few seconds.

Model.create()如果您要处理非常大的批量，这是一种不好的插入方式。它会很慢。在这种情况下，您应该使用Model.collection.insert，它的性能要好得多。根据大小的散装，Model.create()甚至会崩溃！尝试了一百万个文件，没有运气。使用Model.collection.insert它只需要几秒钟。

Model.collection.insert(docs, options, callback)

• docsis the array of documents to be inserted;
• optionsis an optional configuration object - see the docs
• callback(err, docs)will be called after all documents get saved or an error occurs. On success, docs is the array of persisted documents.

• docs是要插入的文档数组；
• options是一个可选的配置对象 - 请参阅文档
• callback(err, docs)将在所有文档保存或发生错误后调用。成功时， docs 是持久化文档的数组。

As Mongoose's author points out here, this method will bypass any validation procedures and access the Mongo driver directly. It's a trade-off you have to make since you're handling a large amount of data, otherwise you wouldn't be able to insert it to your database at all (remember we're talking hundreds of thousands of documents here).

正如 Mongoose 的作者在此处指出的那样，此方法将绕过任何验证程序并直接访问 Mongo 驱动程序。这是您必须进行的权衡，因为您正在处理大量数据，否则您根本无法将其插入到数据库中（请记住，我们在这里讨论的是数十万个文档）。

A simple example

一个简单的例子

var Potato = mongoose.model('Potato', PotatoSchema);

var potatoBag = [/* a humongous amount of potato objects */];

Potato.collection.insert(potatoBag, onInsert);

function onInsert(err, docs) {
    if (err) {
        // TODO: handle error
    } else {
        console.info('%d potatoes were successfully stored.', docs.length);
    }
}

Update 2019-06-22: although insert()can still be used just fine, it's been deprecated in favor of insertMany(). The parameters are exactly the same, so you can just use it as a drop-in replacement and everything should work just fine (well, the return value is a bit different, but you're probably not using it anyway).

2019 年 6 月 22 日更新：虽然insert()仍然可以正常使用，但它已被弃用，以支持insertMany(). 参数完全相同，因此您可以将其用作替代品，一切都应该可以正常工作（好吧，返回值有点不同，但无论如何您可能都没有使用它）。

Reference

参考

• Mongo documentation
• Aaron Heckman on Google Groups discussing bulk inserts

• Mongo 文档
• Aaron Heckman 在 Google Groups 上讨论批量插入

回答by Derek

Mongoose 4.4.0 now supports bulk insert

Mongoose 4.4.0 现在支持批量插入

Mongoose 4.4.0 introduces --true-- bulk insert with the model method .insertMany(). It is way faster than looping on .create()or providing it with an array.

Mongoose 4.4.0 引入了 --true-- 使用模型方法的批量插入.insertMany()。它比循环.create()或提供数组要快得多。

Usage:

用法：

var rawDocuments = [/* ... */];

Book.insertMany(rawDocuments)
    .then(function(mongooseDocuments) {
         /* ... */
    })
    .catch(function(err) {
        /* Error handling */
    });

Or

或者

Book.insertMany(rawDocuments, function (err, mongooseDocuments) { /* Your callback function... */ });

You can track it on:

您可以在以下位置跟踪它：

• https://github.com/Automattic/mongoose/issues/723
• https://github.com/Automattic/mongoose/blob/1887e72694829b62f4e3547283783cebbe66b46b/lib/model.js#L1774

• https://github.com/Automattic/mongoose/issues/723
• https://github.com/Automattic/mongoose/blob/1887e72694829b62f4e3547283783cebbe66b46b/lib/model.js#L1774

回答by benske

Indeed, you can use the "create" method of Mongoose, it can contain an array of documents, see this example:

的确，你可以使用 Mongoose 的“create”方法，它可以包含一个文档数组，看这个例子：

Candy.create({ candy: 'jelly bean' }, { candy: 'snickers' }, function (err, jellybean, snickers) {
});

The callback function contains the inserted documents. You do not always know how many items has to be inserted (fixed argument length like above) so you can loop through them:

回调函数包含插入的文档。您并不总是知道必须插入多少项（固定参数长度，如上），因此您可以遍历它们：

var insertedDocs = [];
for (var i=1; i<arguments.length; ++i) {
    insertedDocs.push(arguments[i]);
}

---

---

Update: A better solution

更新：更好的解决方案

A better solution would to use Candy.collection.insert()instead of Candy.create()- used in the example above - because it's faster (create()is calling Model.save()on each item so it's slower).

更好的解决方案是使用Candy.collection.insert()而不是Candy.create()- 在上面的示例中使用 - 因为它更快（create()正在调用Model.save()每个项目，所以它更慢）。

See the Mongo documentation for more information: http://docs.mongodb.org/manual/reference/method/db.collection.insert/

有关更多信息，请参阅 Mongo 文档：http: //docs.mongodb.org/manual/reference/method/db.collection.insert/

(thanks to arcseldonfor pointing this out)

（感谢arcseldon指出这一点）

回答by SUNDARRAJAN K

You can perform bulk insert using mongoDB shell using inserting the values in an array.

您可以使用 mongoDB shell 通过在数组中插入值来执行批量插入。

db.collection.insert([{values},{values},{values},{values}]);

回答by user2582680

You can perform bulk insert using mongoose, as the highest score answer. But the example cannot work, it should be:

您可以使用 mongoose 执行批量插入，作为得分最高的答案。但是这个例子不起作用，它应该是：

/* a humongous amount of potatos */
var potatoBag = [{name:'potato1'}, {name:'potato2'}];

var Potato = mongoose.model('Potato', PotatoSchema);
Potato.collection.insert(potatoBag, onInsert);

function onInsert(err, docs) {
    if (err) {
        // TODO: handle error
    } else {
        console.info('%d potatoes were successfully stored.', docs.length);
    }
}

Don't use a schema instance for the bulk insert, you should use a plain map object.

不要为批量插入使用架构实例，您应该使用普通地图对象。

回答by Arpit

Here are both way of saving data with insertMany and save

这里有两种使用 insertMany 和 save 保存数据的方法

1) Mongoose save array of documents with insertManyin bulk

1) MongooseinsertMany批量保存文档数组

/* write mongoose schema model and export this */
var Potato = mongoose.model('Potato', PotatoSchema);

/* write this api in routes directory  */
router.post('/addDocuments', function (req, res) {
    const data = [/* array of object which data need to save in db */];

    Potato.insertMany(data)  
    .then((result) => {
            console.log("result ", result);
            res.status(200).json({'success': 'new documents added!', 'data': result});
    })
    .catch(err => {
            console.error("error ", err);
            res.status(400).json({err});
    });
})

2) Mongoose save array of documents with .save()

2) Mongoose 保存文档数组 .save()

These documents will save parallel.

这些文件将并行保存。

/* write mongoose schema model and export this */
var Potato = mongoose.model('Potato', PotatoSchema);

/* write this api in routes directory  */
router.post('/addDocuments', function (req, res) {
    const saveData = []
    const data = [/* array of object which data need to save in db */];
    data.map((i) => {
        console.log(i)
        var potato = new Potato(data[i])
        potato.save()
        .then((result) => {
            console.log(result)
            saveData.push(result)
            if (saveData.length === data.length) {
                res.status(200).json({'success': 'new documents added!', 'data': saveData});
            }
        })
        .catch((err) => {
            console.error(err)
            res.status(500).json({err});
        })
    })
})

回答by ddennis

It seems that using mongoose there is a limit of more than 1000 documents, when using

好像使用mongoose有1000多个文件的限制，使用的时候

Potato.collection.insert(potatoBag, onInsert);

You can use:

您可以使用：

var bulk = Model.collection.initializeOrderedBulkOp();

async.each(users, function (user, callback) {
    bulk.insert(hash);
}, function (err) {
    var bulkStart = Date.now();
    bulk.execute(function(err, res){
        if (err) console.log (" gameResult.js > err " , err);
        console.log (" gameResult.js > BULK TIME  " , Date.now() - bulkStart );
        console.log (" gameResult.js > BULK INSERT " , res.nInserted)
      });
});

But this is almost twice as fast when testing with 10000 documents:

但是在使用 10000 个文档进行测试时，这几乎快了两倍：

function fastInsert(arrOfResults) {
var startTime = Date.now();
    var count = 0;
    var c = Math.round( arrOfResults.length / 990);

    var fakeArr = [];
    fakeArr.length = c;
    var docsSaved = 0

    async.each(fakeArr, function (item, callback) {

            var sliced = arrOfResults.slice(count, count+999);
            sliced.length)
            count = count +999;
            if(sliced.length != 0 ){
                    GameResultModel.collection.insert(sliced, function (err, docs) {
                            docsSaved += docs.ops.length
                            callback();
                    });
            }else {
                    callback()
            }
    }, function (err) {
            console.log (" gameResult.js > BULK INSERT AMOUNT: ", arrOfResults.length, "docsSaved  " , docsSaved, " DIFF TIME:",Date.now() - startTime);
    });
}

回答by xameeramir

Sharing working and relevant code from our project:

分享我们项目中的工作和相关代码：

//documentsArray is the list of sampleCollection objects
sampleCollection.insertMany(documentsArray)  
    .then((res) => {
        console.log("insert sampleCollection result ", res);
    })
    .catch(err => {
        console.log("bulk insert sampleCollection error ", err);
    });