Mongoose (mongodb) batch insert?

by

Model.create() vs Model.collection.insert(): a faster approach

Model.create() is a bad way to do inserts if you are dealing with a very large bulk. It will be very slow. In that case you should use Model.collection.insert, which performs much better. Depending on the size of the bulk, Model.create() will even crash! Tried with a million documents, no luck. Using Model.collection.insert it took just a few seconds.

Model.collection.insert(docs, options, callback)
  • docs is the array of documents to be inserted;
  • options is an optional configuration object – see the docs
  • callback(err, docs) will be called after all documents get saved or an error occurs. On success, docs is the array of persisted documents.

As Mongoose’s author points out here, this method will bypass any validation procedures and access the Mongo driver directly. It’s a trade-off you have to make since you’re handling a large amount of data, otherwise you wouldn’t be able to insert it to your database at all (remember we’re talking hundreds of thousands of documents here).

A simple example

var Potato = mongoose.model('Potato', PotatoSchema);

var potatoBag = [/* a humongous amount of potato objects */];

Potato.collection.insert(potatoBag, onInsert);

function onInsert(err, docs) {
    if (err) {
        // TODO: handle error
    } else {
        console.info('%d potatoes were successfully stored.', docs.length);
    }
}

Update 2019-06-22: although insert() can still be used just fine, it’s been deprecated in favor of insertMany(). The parameters are exactly the same, so you can just use it as a drop-in replacement and everything should work just fine (well, the return value is a bit different, but you’re probably not using it anyway).

Reference

More Related Contents:

  1. Push items into mongo array via mongoose
  2. E11000 duplicate key error index in mongodb mongoose
  3. Mongoose: findOneAndUpdate doesn’t return updated document
  4. Mongoose populate vs object nesting
  5. Stop Mongoose from creating _id property for sub-document array items
  6. Populate nested array in mongoose
  7. Why does mongoose always add an s to the end of my collection name
  8. Cannot overwrite model once compiled Mongoose
  9. Mongoose Schema hasn’t been registered for model
  10. Mongoose findByIdAndUpdate not returning correct model
  11. Mongoose duplicate key error with upsert
  12. How can I use a cursor.forEach() in MongoDB using Node.js?
  13. Mongoose update without callback
  14. Model.find().toArray() claiming to not have .toArray() method
  15. How to fix ‘Error: querySrv EREFUSED’ when connecting to MongoDB Atlas?
  16. How to define object in array in Mongoose schema correctly with 2d geo index
  17. Difference between MongoDB and Mongoose
  18. How to get all count of mongoose model?
  19. How can I save multiple documents concurrently in Mongoose/Node.js?
  20. Mongoose: CastError: Cast to ObjectId failed for value “[object Object]” at path “_id”
  21. Model.find() returns empty in mongoose [duplicate]
  22. How do I increment a Number value in Mongoose?
  23. How to reference another schema in my Mongoose schema?
  24. Mongoose – finding subdocuments by criteria
  25. Mongoose findOneAndUpdate and runValidators not working
  26. Populate a mongoose model with a field that isn’t an id
  27. Mongoose select fields to return from findOneAndUpdate
  28. mongoose recursive populate
  29. MongoDB $lookup vs Mongoose populate
  30. Heroku app crashes after MongoDB updated to 3.0

Leave a Comment