Hacker News Reader

Comment by efbenson | Hacker News Reader

For startup projects I love Mongo because we can get a product up and running very quickly. However I always feared in the back of my head we would have to move off of it if our service got too big. Maybe it is all the complaints from a small portion of heavy users. Regardless, big updates like this are going a long way to help make me feel content on continuing to use it as we grow.

View on HN

> However I always feared in the back of my head we would have to move off of it if our service got too big

I like to call that "nice problem to have" territory. There is nothing to fear from being too successful, or from having the problems of success. Success problems can be solved by the application of people, time and money. And like all optimization you measure first to make sure you really have the problems you think you do, Amdahl's law etc.

Far more likely you'll have the problems of not being successful, such as no attention or money from customers and investors. Or not being in the business you think you are in. There isn't really that much point putting in infrastructure just in case you get successful, if that same infrastructure takes time to develop, and slows development.

I'd be delighted having a service "too big" for MongoDB! After all it would mean being more successful than the companies listed at http://www.mongodb.org/display/DOCS/Production+Deployments

With MongoDB, you are able to shard your system. This means you can grow your databases horizontally. This is not something you can easily (or cheaply) do in the world of RDBMS. You will see much better scalability with MongoDB than with something like MySQL.

You can shard RDBMS relatively easily -- basically you wind up pushing a part of your database structure into your clients, so your clients can decide which shard to use.

The cost, though, is that you wind up having a difficult time doing some things that MongoDB can't do. (For example: Renormalizing your database... Does that even mean anything for MongoDB?)

There's something to be said, of course, for simplifying your design. But it's probably a good idea also to make sure your design reflects your requirements.

Indeed you can distribute data to multiple independent RDBMS, but balancing when new nodes are added is probably a manual process (or a lot of custom code) that is likely to require downtime. To avoid downtime, your application would need to write to both chunks while it is balancing/migrating (and then delete the old data/chunks once it is migrated to a the RDBMS). Essentially, you would need to write what is already in MongoDB.

You would also have to write a parallel query engine.

I too am a fan of simple designs, but I think rolling your own sharding on top of a RDBMS would likely be a massive chunk of time.

There are really expensive commercial products working on horizontally scaling RDBMS... but personally, I prefer open source and document oriented databases :-)

Reply on news.ycombinator.com