MongoDB office hours in Mountain View

You may recall my previous post on MongoDB and how powerful it is as an alternative to a relational database. Since then I’ve had a bunch of discussions with other software engineers around this space and even met up with MongoDB core engineer Chris Westin from 10gen at Red Rock to gain further insights into MongoDB.

New kick ass GUI for MongoDB

Chris introduced me JMongoBrowser written by Antoine Girbal (10gen engineer). It’s written in Java so you can run it on Linux, Windows and Mac OSX. So far this GUI has proven to be a success and fills the holes where MongoHub couldn’t. Out goes MongoHub and in goes JMongoBrowser.

Fast crash recovery using Journaling

MongoDB uses memory 1st to write data to vs directly to file/store. This is where huge performance gains are attained. It also has Journaling, a write-ahead for operations to facilitate fast crash recovery in the storage engine. This means the stuff in memory is stored in a log incase your server goes down without affecting MongoDB’s performance.

So what happens if your box goes down? .. a common question amongst new engineers to MongoDB. Does this data also gets lost? The answer lays somewhere in between how you balance your performance needs against the risks you are willing to take with your data.

The journal is synced to disk every 100ms. So the maximum that can be lost is up to 100ms worth of changes. At the cost of additional performance degradation, you can make your application proof against even that.  The j option to getLastCommand will cause the application to block until the journal entries with the last change have been written to disk.  See http://www.mongodb.org/display/DOCS/getLastError+Command .  Of course, if you use this, your call to getLastCommand can wait up to 100ms, depending on where in the flush cycle you are. Therefore, this is left up to the user to change this default of 100ms.

Therefore, always load MongoDB with Journaling enabled, like this:

mongod --journal

This also auto cleans up crashes and puts crashed data back into MongoDB.

Don’t be alarmed when you see this

I found this in the /Journal (1GB files):

-rw------- 1 root root 1.0G 2011-06-23 02:26 prealloc.0
-rw------- 1 root root 1.0G 2011-06-23 02:25 prealloc.1
-rw------- 1 root root 1.0G 2011-06-23 02:26 prealloc.2

With Journaling enabled the server always creates those three 1GB files.  It rotates through them, recycling them.  They won’t grow any more.  But they are always that size, regardless of the size of your database. If the server dies unexpectedly, the files remain, and contain the material necessary for the automated recovery that happens when you restart the server.

More here:

• Journaling: http://www.mongodb.org/display/DOCS/Journaling
• Durability and Repair: http://www.mongodb.org/display/DOCS/Durability+and+Repair

Checking server memory usage

As mentioned above, Memory is used by Mongo to speed things up. The more memory you have the better and MongoDB will use your RAM as it sees need for it taking into consideration other server resources.

It’s always a good practice to check memory usage.
Details here: http://www.mongodb.org/display/DOCS/Checking+Server+Memory+Usage

Your MongoDB configuration file

Here’s a recommend set of switches to have enabled in your mongodb.conf.

sudo nano /etc/mongodb.conf

add or update your settings to these:

journal=true
directoryperdb=true
logappend=true

• journal = as discussed above to enable fast recovery from crashes.
• directoryperdb = creates a new physical directory for each new database. Clean way to seperate your databases.
• logappend = Whether the log file will be appended or over-written. Always have true else after a reboot your old logs will be overwritten and you may lose important crash specific data.

More detail here: http://www.mongodb.org/display/DOCS/File+Based+Configuration

MySQL to MongoDB

Finally, I found this fantastic chart illustrating the difference in commands between MySQL and MongoDB. Should help the transition for us MySQL folks. Click the image below to download a large PDF version (Size: 213Kb).

There is also mapping chart SQL to Mongo located on the MongoDB website here.

Have more questions? Attend the weekly MongoDB Office Hours in Mountain View Red Rock: http://www.meetup.com/San-Francisco-MongoDB-User-Group/events/16985746/ or post a question/comment below.

~ Ernest