How can I save multiple documents concurrently in Mongoose/Node.js?

Solution 1:

Mongoose does now support passing multiple document structures to Model.create. To quote their API example, it supports being passed either an array or a varargs list of objects with a callback at the end:

Candy.create({ type: 'jelly bean' }, { type: 'snickers' }, function (err, jellybean, snickers) {
    if (err) // ...
});

var array = [{ type: 'jelly bean' }, { type: 'snickers' }];
Candy.create(array, function (err, jellybean, snickers) {
    if (err) // ...
});

Edit: As many have noted, this does not perform a true bulk insert - it simply hides the complexity of calling save multiple times yourself. There are answers and comments below explaining how to use the actual Mongo driver to achieve a bulk insert in the interest of performance.

Solution 2:

Mongoose 4.4 added a method called insertMany

Shortcut for validating an array of documents and inserting them into MongoDB if they're all valid. This function is faster than .create() because it only sends one operation to the server, rather than one for each document.

Quoting vkarpov15 from issue #723:

The tradeoffs are that insertMany() doesn't trigger pre-save hooks, but it should have better performance because it only makes 1 round-trip to the database rather than 1 for each document.

The method's signature is identical to create:

Model.insertMany([ ... ], (err, docs) => {
  ...
})

Or, with promises:

Model.insertMany([ ... ]).then((docs) => {
  ...
}).catch((err) => {
  ...
})

Solution 3:

Mongoose doesn't have bulk inserts implemented yet (see issue #723).

Since you know the number of documents you're saving, you could write something like this:

var total = docArray.length
  , result = []
;

function saveAll(){
  var doc = docArray.pop();

  doc.save(function(err, saved){
    if (err) throw err;//handle error

    result.push(saved[0]);

    if (--total) saveAll();
    else // all saved here
  })
}

saveAll();

This, of course, is a stop-gap solution and I would recommend using some kind of flow-control library (I use q and it's awesome).

Solution 4:

Bulk inserts in Mongoose can be done with .insert() unless you need to access middleware.

Model.collection.insert(docs, options, callback)

https://github.com/christkv/node-mongodb-native/blob/master/lib/mongodb/collection.js#L71-91

Solution 5:

Use async parallel and your code will look like this:

  async.parallel([obj1.save, obj2.save, obj3.save], callback);

Since the convention is the same in Mongoose as in async (err, callback) you don't need to wrap them in your own callbacks, just add your save calls in an array and you will get a callback when all is finished.

If you use mapLimit you can control how many documents you want to save in parallel. In this example we save 10 documents in parallell until all items are successfully saved.

async.mapLimit(myArray, 10, function(document, next){
  document.save(next);
}, done);