When it comes to ODM layer for Node.JS with MongoDB, Mongoose is the top pick for most of the people. With its wide usage it is quite stable and have almost all the features you need. During my recent usage of Mongoose I came across a bug which exists for quite a while and by the looks of it might take much longer to get it fixed. So here is my take on it.

Issue

For the bulk creation of documents Mongoose provides a method on the Model called create(). Mongoose docs gave a good overview of what it does. It takes an array of docs and will call save on each of them individually. Looks fair enough to use it for some bulk docs creation until you ran into the issue wherein one of the doc fails to save. From general thinking I expected Mongoose to return an array which holds the created docs and errors for any doc failed. But it turns out Mongoose will throw the first error it has encountered leaving us with no clue on how many docs were created or errored.

Trials

Since native function didn't work as I expected it to, I set out to do the bulk operations on my own. My initial thought is to call save on each doc myself and batch up the results

function userBulkSave(users) {
  return Promise.allSettled(users.map(user=> new User(doc).save()))
}

I've used Promise.allSettled() instead of Promise.all() so that my function won't return immediately on first failure like the Mongoose create() method. But there are couple of issues here

1. Promise.allSettled() is not natively supported in Node until 12.9. Since the stack I'm working with must also run under older versions of node, this is a roadblock for me. I can use a shim to let the older versions of Node support this functionality but ran into another hiccup as provided below.
2. Even if Promise.allSettled() is supported the resulting array from it won't contain creation responses directly but will be present as {status: "fulfilled", value: {...}} which is not what I want

Possible Solution

So I've got back to drawing board and tried to implement it in an another way using the functionality already existing in the Node

function userBulkSave(users) {
  return Promise.all(
        users.map(user => {
            return new Promise(resolve => {
                new User(user)
                    .save()
                    .then(resp => {
                        resolve(resp);
                    })
                    .catch(err => {
                        resolve(err);
                    });
            });
        }),
    );
}

This seem to do the work for me although with some caveats. Resulting array doesn't have any flag which specifies whether save is successful or not. In this case, since all newly created docs will have an _id in them I kind of used its presence to understand whether the save is successful or not.

Improvisation

Since this can be used in multiple places, instead of exporting this function, we can create this as a statics function under the Mongoose Model so that we can just use it like any other methods provided by Mongoose.

userSchema.statics.bulkSave = function(users) {
    return Promise.all(
        users.map(user => {
            return new Promise(resolve => {
                new this(user)
                    .save()
                    .then(resp => {
                        resolve(resp);
                    })
                    .catch(err => {
                        resolve(err);
                    });
            });
        }),
    );
};

Major changes here is the use of this instead of Model name before. This is because Mongoose will make the Model available to the statics as this.

Since we might need such bulkSave for many other models we can create a generic function and append under statics method under all required models.

const bulkSave = function(docs) {
    return Promise.all(
        docs.map(doc => {
            return new Promise(resolve => {
                new this(doc)
                    .save()
                    .then(resp => {
                        resolve(resp);
                    })
                    .catch(err => {
                        resolve(err);
                    });
            });
        }),
    );
};

UserSchema.statics.bulkSave = bulkSave;

Some points to note

• I've thought to use create() instead of insertMany() so that all my middleware's would be called for the new docs.
• Even though the solution will work for couple of hundreds of docs its not a scalable solution for heavier loads. We might need to add some guard code to send only a couple of hundreds of request at a time to prevent overloading the service and db.