Hacker News Reader

Comment by cmer | Hacker News Reader

My experience with MongoDB hasn't been the most pleasant in a write-heavy environment. Until they fix the write lock properly, MongoDB is pretty much useless for many high throughput applications in my opinion...

The new DB level locking introduced in this release is a joke. There's not much difference between that and the old global write lock unless you split your database in dozens of smaller ones. What a pain. I wish they'd just stop pretending and addressed the issue properly once and for all.

I really want to like and use MongoDB because the way data is represented and how it can be queried is awesome.

View on HN

It's inaccurate that there's no difference between the global lock and database level locking. Whilst it's true that locking is now down at the database level and you get benefits from splitting into multiple databases, the real benefit is from the new PageFaultException architecture. Even using a single database you will see significant performance improvements.

I benchmarked this at http://blog.serverdensity.com/goodbye-global-lock-mongodb-2-... and there is a video explaining how this works at http://www.10gen.com/presentations/concurrency-internals-mon...

That's good to know. It still feels like an half-assed solution to a very serious problem, however.

I definitely won't be looking at MongoDB again until this is fixed for good, and until I know replication is more reliable. Had terrible problems that that too unfortunately.

It might "feel" like a bad solution not granular enough but if you actually test the implementation then locking is no longer a real concern. There are still improvements to be made and this is the first step towards that. Changing from a global to database lock required huge amounts of work which sets the foundations for the next releases with collection and document level locking.

In my experience replication has been extremely stable and reliable for several point releases so I'm not sure what problems you had.

It's not half-assed so much as a first step. 10gen have previously stated that the plan is to gradually increase the granularity or write-locks over successive releases.

Until they end up with MVCC...

aggregation and then MVCC.. the circle is complete.

One of the reasons DB-level locking is actually quite important is that the replication oplog is on the 'local' database and your data is on other databases -- so your application is no longer fighting with replication for the write lock.

David knows what he is talking about, they use MongoDB in really high scale. Also, getting database level locking was like 90% of the work according to the CEO. Doing collection and finally document level locking should be somewhat trivial now. 10Gen just want's to take it slow, and make sure database level locking is working 100% correct before moving onto more granular locking.

Yep they sure are taking it nice and slow... at this pace, collection-level locking is probably only 2 more years away. Other nosql databases are nipping at their heels.

David may use MongoDB at really high scale, but since when is that a reason to call someone an expert.

Moving fast and breaking things in the database world is not a good thing. That said... I highly doubt collection level locking will take too long.

I don't see anyone nipping at their heels. They have some competition in the key/value space, but there are a few different KV paradigms.

MongoDB is a complete replacement for most RDBMS apps and they are way ahead of the pack.

DB Level Locking (https://jira.mongodb.org/browse/SERVER-4328) is a dependency of Collection Level Locking (https://jira.mongodb.org/browse/SERVER-1240). For a critical, tricky core piece of functionality like this it makes sense to tackle one major concurrency issue at a time, do a release, stabilise the code and verify there are no issues introduced by the new locking code and only then release the next step in locking enhancements. Doesn't seem like they're "pretending" about anything as far as I can tell.

They were certainly pretending like it wasn't an issue when I contacted their support some time ago.

Support? Pretending something isn't an issue? Must be entirely unique to this technology company alone.

Actually, there is a lot of difference but not just because of the global lock to database lock transition. There is also a lot of work done on yield-before-lock: http://blog.serverdensity.com/goodbye-global-lock-mongodb-2-... And don't forget that replication also happens in a different database...

The problems with early rev SQL servers were similar (in inconvenience) but, with feedback from users and hard work from developers, they soon moved beyond MVP.

I'm willing to suck up some inconvenience with MongoDB, it's one of the most inspiring technologies of the last decade. Web application persistance feels more like a fully connected limb now.

I can understand the gripes but do urge people to give 10gen some slack and appreciate that what they are building takes time - it's clear that the team are working hard on features driven by community feedback.

It's more than inconvenience when your whole service is down because some database queries take up to 30 seconds to complete (and often time out), and when replication on some servers is days behind (and some are up to date). We also had to completely flush our data a few times because of some corruption we couldn't recover from.

The truth is, MongoDB is awesome if your dataset fits in memory, but if you're in a write-intensive environment with 400-500GB of data, it's just not there yet.

Not true. We're processing 12TB of data each month with thousands of writes per second and sub-millisecond response times. You simply have to understand how to use MongoDB correctly which isn't difficult with the out of the box settings and by reading some of the documentation. Figuring out your working set is the most difficult part but that doesn't take long to calculate based on understanding the queries you're doing and creating the appropriate indexes.

Since you seem to be fairly negative about MongoDB but light on details, perhaps you should write up your experiences so others can learn from what you did wrong.

I have used Mongodb a bit, but only for applications that mostly need to read a lot. What issues might I run into if I need to do lot of writing as well?

None, almost certainly -- if you have enough RAM to fit your working data set, then MongoDB's writes are just changing memory. If it does need to read data from disk before changing it, the new yielding architecture that others have mentioned will help prevent lock contention.

Reply on news.ycombinator.com