Hacker News Reader

Comment by Pxtl | Hacker News Reader

> It is also very easy to use and manage.

This is something that SQL database fans don't get. A small start-up can't afford to have a team of guys fiddling with incredibly obscure performance tuning settings.

A developer might not know how to optimize a slow-running Postgre query, but he knows how to cache data and denormalize in a manner that makes the query he wants to run fast. And a document-store plays nice with this.

Any simple relational beauty of SQL databases falls down once you get over a few million rows, and then the hideous hacks and bizarre settings start coming out. At that point you need a pile of domain-specific knowledge to make them play nice.

Mongo takes the domain-specific knowledge out of the equation and lets the developers apply their existing skill-set to the data.

I've you've got a thousand employees? Get a few DBAs and Postgre.

If you've got, like, five?

View on HN

A small startup that invests in some ops talent is going to run rings around a small startup that optimizes for developers.

People think that staffing up devs creates agility and speed, when in reality it just increases product features.

Scope creep without someone around to voice the needs of operations is a crazy-efficient generator of technical debt.

One thing you don't know or faced yet is that mongo fails, and sometimes it fails so hard that you would rather kick some rocks.

For example, if you start a background index generation on an eventually consistent replica set, indexing on secondary nodes are done foreground. Which means you only accept reads from slaves but slaves are unresponsive because of the index generation. In this state, if you try to do anything fancy your data will go corrupt. Only way out is to wait through the outage (which I find it pretty hard to do so). This is still not solved in 2.4, waiting for 2.6.

Replica sets with all secondaries which can't elect a primary because it lost a node, or the mostly random primary-secondary switches that drops all connections, seldomly primary reelecting itself meanwhile dropping connections for no apparent reasons. Mongo offers tin foil hats for integrity, consistency and reliability. So yeah, I'd rather examine and understand why an SQL query is slow. Because it is at least deterministic, which in mongo nothing really is.

Postgres supports free form json, XML or hstore document formats by the way, couchDb has its own specific features as a document db too. I still don't see why people want to go on with mongo this bad.

I've been running, on a five minute cron, kill -9 on the master mongo instance in our QA lab for a good long time. 24/7.

There's a program running that, 100 times per second, reads a document, MD5's the field, and writes it back. At the same time, it reads a file from the local filesystem, MD5's it, and writes it back. The document and the local filesystem file started with the same value.

After a few thousand kill -9's on the master instance, the local file and the mongo document are still identical.

I've been running MongoDB in production since 2010.

It's definitely possible to use Mongo in a way that isn't safe for your particular use case. But we're doing it correctly.

I haven't lost a bit of data in more than three years of MongoDB.

Mongo has a lot of limitations. We're currently researching various 'big data' solutions, because for us, Mongo doesn't fit that.

For ease of development (in dynamic languages, where your in-program data structures and in-database documents look almost identical), safety, and lack of headaches, MongoDB has been a consistent win for me and the teams I've been on.

Is your test producing a new document value 100 times a second, or just writing the same value back over and over again?

It sounds like it might be the latter, which is not a particularly stressful test (because you can't detect data rollback).

I'm more familiar with relational database internals, but I wouldn't be surprised if a DB just optimized out the unchanged-write entirely (they'd still need to read the current row value, but they don't have to invoke any data modification code once they see the value hasn't changed).

For a good test, you really want to simulate a power-loss, which you aren't getting when you do a process-level kill, because all the OS cache/buffers survives. You can do simulate this with a VM, or with a loopback device. I'd be amazed if MongoDB passed a changing-100-times-a-second test then. I'd be amazed if any database passed it. I'd even be amazed if two filesystems passed :-)

I'm reading the document, hashing it, and writing the hashed value back. So it changes every time.

I plan on extending this test by blocking 27017 with iptables, then doing the kill -9, then wiping out all of the database files. That'll be fun. :)

"A developer might not know how to optimize a slow-running Postgre query, but he knows how to cache data and denormalize in a manner that makes the query he wants to run fast"

And you can cache and denormalize with postgres just as easily, and it will probably perform better and not corrupt your data.

What mongo makes easy is things like replication and failover. It can go horribly wrong on that front, but until you see that, it is much easier to get up and running with replication and failover.

No, with Mongo you get a bunch hacks you need to do to maybe get indexed reads. I had to write a query optimiser that does lots of crazy things and simply rejects some entirely reasonably-looking queries because Mongo can't do them efficiently.

Not to mention the complete lack of durability. Even Redis is more durable.

Postgres is trivial to use. Is something slow (which will happen much later than with Mongo)? Add an index, done. I'm not a DBA, you don't need one.

are there no problems with mongo 'once you get over a few million rows'?

Reply on news.ycombinator.com