Goodbye global lock – MongoDB 2.0 vs 2.2 | Hacker News Reader

Goodbye global lock – MongoDB 2.0 vs 2.2

blog.serverdensity.com

79 pointsbsg7534 comments

They've moved from a _process_ level lock to a _database_ level lock. Let's take a moment to golf clap their mediocrity. golf clap

Use a real database.

If you take a look at the benchmarks further down the page, the real improvement seems to be that mongo does a better job of yielding during page faults. That provides an improvement even with a single database.

I am a Real Database(tm) person, but I will give them some credit for:

"Nevertheless, a major focus for the upcoming 2.2 release has been removing the global lock and introducing database level locking as an initial step towards collection level locking and potentially even more granular concurrency in future releases."

This stuff is hard to get right, and why RDBMS' are simultaneously maligned as "old tech", yet are still the most dependable database mechanisms available.

It seems like the successful(popular) "NoSQL" engines will eventually reinvent many RDBMS wheels - at least in the cases where they are aiming to be replacements.

> It seems like the successful(popular) "NoSQL" engines will eventually reinvent many RDBMS wheels.

The wheels in question aren't RDBMS wheels, they're just generic database wheels. The backlash against RDBMSs wasn't because of implementation details like this, it was because of the baggage of relational structures in general, and SQL in particular.

The "reinvention" of these wheels is occurring now because non-relational databases are maturing now.

Point taken.

As I spend most of my time in SQL, and have come to find a comfortable language to use, I can still understand why fixed schemas give developers headaches - especially when schema migrations in MySQL can result in extra work to minimize downtime.

I mention MySQL specifically, because most of the RDBMS -> NoSQL cases I see came from MySQL users. Coming from MSSQL, and now PostreSQL, it seems as though NoSQL is partially a NoMySQL sentiment.

I don't claim that it is easy, especially with an RDBMS. Actually, given the few courses I took in university on database design, I know its very non-trivial.

But MongoDB is not an RDBMS. There are no joins, so a query can only touch one collection (table) at a time anyways, so at least in that regards its much easier than an RDBMS.

True - s/RDBMS/DBMS/g

It seems like Mongo in particular is reinventing the RDBMS wheel entirely, except with a frustrating JSON query language. The benchmark blog post a couple of days ago versus an out-of-the-box Postgres deploy (i.e. memory crippled) was not very impressive.

And look: they've implemented clusters - by pushing a Slony equivalent into the DB itself.

Please forgive my bile - I was a little nonplussed with Mongo, and now I have to use it for a client. Now I'm a lot nonplussed.

Just an FYI - you're using the word "nonplussed" incorrectly.

Perhaps shubber is nonplussed after learning about MongoDB's query language design and the client's choice of database -- why would anyone actually want such a monstrosity?!

Because MongoDB is a fantastic database for developers whilst RDBMS can suck big time.

In this age of agile development requirements and schemas change very, very rapidly and MongoDB excels at being able to support that.

I am genuinely interested in why schema changes in RDBMS' are such a point of contention. Is it the addition of new columns that are problematic, or the modification of existing column datatypes?

Having written more than one mechanism to automate schema upgrades in remote deployments, and numerous migration scripts, I know it is an extra step in the development process, but I have not considered it an onerous one.

For me, the lack of schema also means that during an upgrade on a large database I can upgrade my app, start batch updating the data in the database and then also perform on-access upgrades and minimize my app downtime. Changing a schema in a large database can potentially require a lot of downtime or other interesting gymnastics to keep downtime to a minimum. I'm not completely sold on Mongo but this is one area where not having a schema can really help.

Would not any online schema update in a more complicated cases (not just simply adding or removing columns) require interesting gymnastics or downtime in MongoDB too? The application needs to support both the old and new (and possible an intermediate) schemas during the online upgrade.

PostgreSQL supports transactional schema modifications. fast adding and removing of columns, and lockless index creation. So in simple cases upgrading the schema is trivial. For complicated cases it can be a mess, but my guess is that that applies to any database.

Schema changes are exclusively an application-layer change in MongoDB. So MongoDB can support as many schemas as you want at the same time whilst you are migrating.

Also don't forget that MongoDB can have arrays and sets as a "column" type. Which if you tried to replicate in RDBMS would mean a multi-table migration.

> Schema changes are exclusively an application-layer change in MongoDB.

Which is not by itself an advantage. You still need to write the code which does the schema change if you for example rename a field.

> Also don't forget that MongoDB can have arrays and sets as a "column" type. Which if you tried to replicate in RDBMS would mean a multi-table migration.

So can PostgreSQL. The sets are not as general as in MongoDB though since they can only store strings.

Try ALTER TABLE ADD COLUMN on a 100 million rows table :).

Jeremy Zawodny from Craigslist explained why it helped them a lot for Craigslist archives database, where an alter table could take up to 24 hours: http://www.10gen.com/presentations/mongosf2011/craigslist

OK, a MySQL problem. Not so much with PostgreSQL, where adding new columns noes not need to touch every table row.

NoSQL ~= NoMySQL

Simple. To add a new attribute with MongoDB I simply add a new variable in my Java model and that's it. Done.

With RDBMS. I have to write an update SQL script, rollback SQL script, run it against my dev environment, ensure it is applied through test environments and hope the DBA doesn't forgot to run it in production (it happens).

I use an RDBMS with Django. When I want to add a new attribute to a schema, I just add the variable to my model, and South picks up on the change and writes the update script for me.

MongoDB gets points for including a query language (vs CouchDB), but loses a few for it being JSON/JavaScript.

I understand the choice, JSON and JavaScript being so popular in current web platforms, but the SQL developer in me wishes for something more "readable" (see Lua in Redis).

RDBMSes still do not provide easy support for sharding. That alone makes MongoDB different.

AFAIK, the locking in Mongo has more to do with them using memory mapped files than anything else. Other DB's, like Riak, do not have this problem. But perhaps someone can prove me wrong.

No need to be snarky, but I certainly share your general sentiment:

http://news.ycombinator.com/item?id=3412386

And they've indicated that in the next stable release they will have collection level locking. All moving towards sub collection locking.

It is a relatively new database and they trying to make changes in small increments to ensure they don't break anything.

Seems prudent to me.

Not that new, and with that much funding they received. They're just being lazy.

Well done 10gen peeps! Keep up the good work. You deserve all the credit you can get. There's still a lot of work to do to catch up with established RDBMSes but you're definitely getting a bunch of things right already today.

i guess then in some respects - its funny how correct Adam was in his comments

http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use...

"10gen do not provide official benchmarks because they tend to be irrelevant to real world usage."

Anybody else thing this sentence is silly? I definitely do not base my judgement on a tool off its artificial benchmarks but I do check them to see if an upgrade is worth it or not. Irrelevant or not, please publish them.

I'm a mongo user(for offline analytics) so any improvement in performance will be good.

>Anybody else thing this sentence is silly?

Not really. It's a lose/lose proposition to publish benchmarks, especially on something that's so environment and dataset dependent. Either real world performance is way below the benchmark leading to all kinds of storm and strife, or way above and then you lose credibility and people wondered why you bother anyways.

Damn, when I glanced upon the title I thought it was about Python's GIL.

What actual use case do you have that CPython's GIL is affecting in any serious way? And if you do have a real case which somehow requires the removal of the GIL, why don't you just use Jython, which has no GIL and recently released an implementation of Python 2.7?

Um, pretty much all parallelizable CPU number crunching (data mining algorithms, computer vision etc.) which aren't covered in C code. Those are especially problematic, since they're usually coupled with numpy and simillar libraries not supported on other Python implementations - including Jython.

These posts (the one about 20ms for a memcached lookup especially) make me doubt the technical prowess of the entire ServerDensity staff, and thereby stopping me from ever considering this product.

Reply on news.ycombinator.com