Using the same MongoDB database with Grails and non-Grails applications

In one of our projects we had reason to combine a Grails-based web application with non-web applications. We were using SpringSource's MongoDB GORM plugin [1] for our Grails-based application. We also had applications that operated on the same database as the web application, for these we used the official MongoDB Java driver [2].

By default, Grails uses a sequence of increasing java.lang.Long-type values as the database identifier of it's Domain objects. One easy way to change the type of the "id" property of your Domain object is to explicitly declare "id" as a different type. If you wish to use a Mongo database in combination with your Grails application as well as other applications, it may be a good idea to set the relevant Domain objects' "id" property to be of type org.bson.types.ObjectId, as this is the default type that the official MongoDB drivers will use to set the "_id" field [3]. This would also scale better when one uses database clusters. However, if you wish to use the standard Grails database identifer, or the project is already well underway by the time you have reason to have other applications operate on the same database, changing the type of the database identifier may not be an option.

If you have both a Grails project with a Long as the identifier type and a project using the official Java driver inserting to and retrieving from the same collections, you will get a mix of both identifier types. This will most likely lead to problems, perhaps with your Grails project, as GORM will encounter ObjectId-based field values that it will try to convert to a Long.

To solve this problem from the non-Grails side we need to make sure that, when inserting a document, the "_id" field of the document is a Long value larger than the largest already present for this Collection. We might want to have a function that reserves the id for us, and makes note of this in a special Collection. This sort of solution is called 'optimistic locking'.

The MongoDB documentation offers a general solution for when one wants to use an increasing number [4], which uses the atomic findAndModify action. One downside to using findAndModify is that you will have to create a Collection and initial values for these identifier bookkeeping documents.

We decided to take a look at how SpringSource's MongoDB GORM plugin solves this problem. The MongoEntityPersister class [5] has a method called "generateIdentifier" which, as expected, generates new identifiers. The class knows if the relevant Collection has numerical identifiers, if not it will assume a BSON ObjectId should be used. In case a numerical identifier is in use, either a second Collection will already have been created, or it will be created by the generateIdentifier method. This Collection will have the name collectionName+"."+NEXT_ID_SUFFIX, which at the time of writing would result in collectionName+".next_id". It will consist of documents of the form '{ "_id" : NumberLong(1) }'.

One might use a function like the one shown below, which is known to be valid Groovy, but looks to be valid Java as well. This particular implementation lacks error handling, and uses a hardcoded partial Collection name.

protected Long generateNewIdentifier(String collectionName){

DBCollection dbCollection = mongoDBDatabase.getCollection(collectionName + ".next_id");

DBCursor dbCursor = dbCollection.find().sort(new BasicDBObject("_id", -1)).limit(1);

long nextId;
if (dbCursor.hasNext()){
final Long current = dbCursor.next().get("_id");
nextId = current + 1;
} else {
nextId = 1;
}

BasicDBObject nextIdObject = new BasicDBObject("_id", nextId);
dbCollection.save(nextIdObject);

return nextId;
}

It would be better if we could make use of the actual NEXT_ID_SUFFIX member from MongoEntityPersister, rather than hardcoding the suffix as we did in the above code sample. Preferably we would retrieve it from the class only once during the lifespan of the application. Retrieving it from the class does require a few additional jar files. When using Spring framework version 3 it will require the jar files with the following Maven artifactIds: grails-datastore-core, grails-datastore-mongo, spring-data-commons-core, spring-data-mongodb, spring-core, spring-context, spring-beans and finally the spring-transaction. Of course, these should be kept up to date with the Grails-based application's releases - if we did otherwise we would still run the risk of using a different bookkeeping Collection than the Grails-based application does.

As we can see, there are a couple of different routes you can take when you run in to this kind of scenario. In our case the most obvious solution, explicitly setting the Grails Domain object's "id" member to be of type org.bson.types.ObjectId, did not seem the best solution because it would mean having to implement additional changes to our Grails web application. This came to be because we had not considered that there might ever be a reason to change our database identifiers from type Long into another type. However, this did not turn out to be a problem - the solution outlined in this post has turned out to work quite well in our scenario.

Tags