mongoose

Update, ElementMatch and the $ Positional Operator on MongoDB/Mongoose

One of the biggest advantages of mongodb is the option of defining documents inside documents and by doing so creating powerful and at the same time flexible data structures.
In most cases, the deepest you will go is one level, even though mongo doesn’t set any limit on how many levels you can go inside a document, for example: A BlogPost can have an array of Comments, or an Image can have an array of Tags, and there is plenty of documentation online on how to manipulate single dimension arrays in documents. However, what if you have a multi dimensional array in a document?
Take for example the following model:

   
 /**  
 * Line Item schema  
 */  
 exports.EstimateLineItem = (function () {  
 schemas.lineItem = new Schema({  
 ‘name’ : String,  
 ‘description’ : String,  
 ‘quantity’ : Number,  
 ‘cost’ : Number  
 });

 return db.model(‘EstimateLineItem’, schemas.lineItem);  
 })();

/**  
 * Estimate schema  
 */  
 exports.Estimate = (function () {  
 schemas.estimate = new Schema({  
 ‘name’ : String,  
 ‘quoteID’ : Number,  
 ‘subTotal’ : Number,  
 ‘finalTotal’ : Number,  
 ‘creationDate’ : { type: Date, default: Date.now },  
 ‘status’ : { type: String, default: "Active" },  
 ‘lineItemSet’ : [schemas.lineItem]  
 });

 return db.model(‘Estimate’, schemas.estimate);  
 })();

/**  
 * Job schema  
 */  
 exports.Job = (function () {  
 schemas.job = new Schema({  
 ‘name’ : String,  
 ‘description’ : String,  
 ‘creationDate’ : { type: Date, default: Date.now },  
 ‘status’ : { type: String, default: "Active" },  
 ‘scheduledDates’ : [Date],  
 ‘customerID’ : ObjectId,  
 ‘estimateSet’ : [schemas.estimate]  
 });

 return db.model(‘Job’, schemas.job);  
 })();

So a Job has an array of Estimates and an Estimate has an array of LineItems.
The problem is, how to add a new LineItem to an Estimate?
One option would be to query for a job, then loop through its estimateSet and find the selected estimate, then push a new line item to the lineItemSet and finally save back the job. However, there is a simpler way:

   
 Job.update(  
 {estimateSet: {"$elemMatch": {_id: estimateID}}},  
 {$push:  
 {  
 "estimateSet.$.lineItemSet":  
 {  
 ‘name’ : lineItem.name,  
 ‘description’ : lineItem.description,  
 ‘quantity’ : parseInt(lineItem.quantity),  
 ‘cost’ : parseInt(lineItem.cost),  
 ‘_id': lineItem._id  
 }  
 }  
 },  
 {upsert:false,safe:true},  
 function (err) {  
 console.log("err: ", err);  
 if (err) {  
 res.send({  
 "err": true,  
 });  
 }  
 else {  
 res.send({  
 "err": false,  
 })  
 }  
 }  
 );

Breaking down the query:
It finds a job that has an Estimate with the specified ID:

{estimateSet: {"$elemMatch": {_id: estimateID}}}

Once the job is found, it pushes a new line item to the lineItemSet:

 
 {$push:  
 {  
 "estimateSet.$.lineItemSet":  
 {  
 ‘name’ : lineItem.name,  
 ‘description’ : lineItem.description,  
 ‘quantity’ : parseInt(lineItem.quantity),  
 ‘cost’ : parseInt(lineItem.cost),  
 ‘_id': lineItem._id  
 }  
 }  
 },  

For more info:

MongooseJS Validators - Contributing to an open source project

Today I was able to put in practice all the tools I learned last semester in the DPS 909 – Topics in Open Source Development class

The Problem

In a project for a class I’m taking this semester, I was working on writing the validation portion for the mongoose schemas for one of the collections being used

   
 User = new Schema({  
 ‘username': {  
 type: String,  
 validate: [validateUsername, ‘username not valid’],  
 },  
 });  
 [/sourcecode]

It’s defening a validator for username, so when saving a User object:

[sourcecode language=”javascript”]  
 var user = new User();  
 user.save(function(err) {

}  

It will call validateUsername on username and if the validation fails the object won’t be saved and the err will have the information about the error.

 
 { message: ‘Validation failed’,  
 name: ‘ValidationError’,  
 errors:  
 { username:  
 { message: ‘Validator "username not valid" failed for path username’,  
 name: ‘ValidatorError’,  
 path: ‘username’,  
 type: ‘username not valid’  
 }  
 }  
 }  

So my problem was that I wanted to add more than one validator to a single field

 
 User = new Schema({  
 ‘username': {  
 type: String,  
 validate: [validateUsername, ‘username not valid’], [validator2, ‘second validator’],  
 },  
 });  

However that didn’t work.
I went back to the mongoose documentation but couldn’t find a way to attach two validators to a single field.

So I was faced with two options, accept the facts and move on, or try to modify the library
Mongoose is an open source library, so I started reading the source code trying to find a way to accomplish my goal.

Because the source code is very organized and easy to read it didn’t take me long to find where a shchema field was being created.

A schema field is defined in the schematype.js

 
 function SchemaType (path, options, instance) {  
 this.path = path;  
 this.instance = instance;  
 this.validators = [];  
 this.setters = [];  
 this.getters = [];  
 this.options = options;  
 this._index = null;

 for (var i in options) if (this[i] && ‘function’ == typeof this[i]) {  
 var opts = Array.isArray(options[i])  
 ? options[i]  
 : [options[i]];

 this[i].apply(this, opts);  
 }  
 };  

path is the name of the field
options is an object cointaining all the options for the field
instance is the type of the field
for example:

 
 path: username  
 instance: String  
 options: {  
 type: [Function: String],  
 validate: [ [Function: validateUsername], ‘username not valid’ ],  
 }  

The for loop iterates through all the options and calls the appropiate function depending on the property
So using the example above, ‘i’ would be equal to ‘validate’.
So calling this[i].apply(this, opts)
would be the same as calling
this.validate([Function: validateUsername], ‘username not valid’)

Here is the part where the validator gets added to the field

 
 SchemaType.prototype.validate = function (obj, error) {  
 this.validators.push([obj, error]);  
 return this;  
 };  

Pretty straight forward, it pushes the function and the error to the validators array.
But I wanted to pass more than just one function and error.

The Solution

 
 SchemaType.prototype.validate = function (obj, error) {  
 if (‘function’ == typeof obj && ‘string’ == typeof error) {  
 this.validators.push([obj, error]);  
 }  
 else {  
 for (var i in arguments) {  
 this.validators.push([arguments[i].func, arguments[i].error]);  
 }  
 }  
 return this;  
 };  

So if I defined more than one validator for a single field in the schema, the arguments var for the validate method would look something like this:

 
 arguments: { ‘0’: { func: [Function: trim], error: ‘trim error’ },  
 ‘1’: { func: [Function: validateEmail], error: ‘email error’ } }  

However I couldn’t just break the existing code, so I added a check to see if there was more than one validator before pushing to the validator array

In the end, the solution worked.
So I created a patch and opened a ticket on the mongoose repo to discuss the issue.
I’m not sure if the change will be accepted in the project, but it was nice to see that I can actually modify the library if nedded

**After going over one more time through the documentation I found a different way to add multiple validators

 
 User.path(‘username’).validate(function (v) {  
 return false;  
 }, ‘my error type’);  
 User.path(‘username’).validate(function (v) {  
 return true;  
 }, ‘another error’);  

It calls the validate function explicity on the field, allowing multiple validators to be added