Why I am dropping MongoDB for Mysql... | Hacker News Reader

Why I am dropping MongoDB for Mysql...

blue74.com

35 pointscaptaincrunch46 comments

There's been a few of these posts recently and in every case it's the fault of the author either because they chose the wrong product for their use, because it's not being used as directed or because the unstable versions are being used.

- There are no transactions in databases like MongoDB. That's clear.

- Reported "corruption" and data loss but no reproduce steps. What version? Did the server crash? Have you reported to the developers? We've been using MongoDB for over a year in production having started at v0.9 and now using only stable releases and have never lost any data when we do 5-8k inserts per second.

- No joins. It's a non relational database, what were you expecting?

- Schemaless. Yes - don't make typos / QA your code.

- Unstable replication. We've had a few issues with replication but nothing that can be described as "unstable". Again, a general statement "nothing but problems" without going into specifics.

- Not stable. I refer above with our long usage history. It's unstable if you use the dev versions, as expected.

I don't claim MongoDB is perfect, indeed I've written about several issues we've had[1], but nothing critical or show stopping. It seems it just isn't a right fit for what the author wants.

[1] http://blog.boxedice.com/2010/02/28/notes-from-a-production-...

So...

- Lack of Transactions

- No joins

- Schema-less

If these are important, you really shouldn't have been using schema-less document database.

- Unstable

This is the only real point I can give to you unless you were actually running the bleed edge version. If you were using the trunk/head version you were just asking for the problems you were having.

> If these are important, you really shouldn't have been using schema-less document database.

What does that have to do with his complaints?

The "lack of transactions" is a lack of MVCC. This has nothing to do with NoSQL, RDBMSs, document databases, or schemas. Many NoSQL databases do offer some transactions support ... like CouchDB or the storage in Google's AppEngine.

"No joins" ... we've been taught for years the virtues of normalization / relational data, both in school and throughout the industry, and now we should let go of everything we've been taught. Great, but where are the best practices / patterns? Where are the books on data-modelling?

"Schema-less" ... it's not really schema-less, it's just that the schema is defined at the application layer. Again, where are the best practices for that?

One advantage of MongoDB in combination with something like Rails is said to be the lack of migrations ... I view that as a disadvantage as migrations are a great way of documenting what's changed.

To get back to my point ... don't shout at potential users something akin to RTFM ... educate them instead.

It isn't RTFM as much as do some research before choosing your data store.

The majority of his enumerated gripes aren't unexpected or hidden attributes of mongodb but fundamental considerations when deciding on a data store.

And his data loss issues seem to be related to server reboots according to one of his later comments. MongoDB's docs clearly state that the database must be shut down safely or data loss will happen. (MySQL is the same way, it gets really annoyed at hard reboots)

And in a moment of pure irony, I'm seeing an "Error establishing a database connection" message.

Indeed. At the risk of being overly flippant, I'd say "maybe you should go back." The page is still loading (after several minutes) for me.

I am getting DoS'd. Got admins working on it now, I am guessing someone didn't like my post lol.

And fixed.

Are you sure this is a DoS? Your article got mentioned here and on Reddit.

It sounds like he wanted a SQL database all along, so there's not much insight in this post.

Really? Then why does he say: "In Mongo’s defense- I really like it, and hope to use it in the future when it becomes stable."

Well if one of his primary reasons for going back was that he doesn't like schema-less databases, then it's a fair thing to say that Mongo was never the right choice.

Of course he may have needed to do this project to discover that.

I'd bite the bullet on schema-less, if it were stable.

http://www.mongodb.org/display/DOCS/Use+Cases

The author's project requirements list transaction support. Meanwhile, MongoDB explicitly mentions it is not suited for transactional systems.

I agree, he needed an SQL db from the beginning.

I can't understand why anyone would start a project with Mysql instead of Postgress.

I can: my cheapie hosting service only provides MySql. But beyond that, no. I certainly prefer PostgreSQL at home, and would use it for other projects when I could. MySQL was simply too late to the "ACID party", and it's hard to believe they took data integrity seriously.

E.g. - transactions, isolation, referential integrity and other feature were too long coming. They did provide a free portable DB that ran on Windows early on and parsed SELECT statements, though :-)

> transactions, isolation, referential integrity and other feature were too long coming.

They had that years ago using InnoDB. I remember using it in 2003.

MySQL's big thing over Postgres was replication.

My perception is also that replication was lately the only advantage Mysql had over Postgres in general.

My main problem with mysql is that I have seen too many data corruptions and I can't find anyone who has worked with both systems and doesn't recommend in general pg over mysql.

FWIW, Dead-simple replication is in the current postgres beta and due to enter stable soon.

Here are the docs: http://developer.postgresql.org/pgdocs/postgres/hot-standby....

Oh, and unlike the MySQL replication this implementation can not silently lose or corrupt data.

In fact the above document could also be considered "The missing Manual for MySQL replication". It tells you all the things that MySQL doesn't take care of.

I can't understand why somebody wouldn't use a VPS ;-)

I have a VPS and I can understand that some people don't have a clue how to admin a server. Maybe that's the reason?

I don't think I'm an uber admin. I just try to use so few services as possible and update my shit. Hopefully this is enough.

Having spent the past 30 minutes trying to get this guy's web server to even return anything, I'm thinking he may reconsider...

Was getting DoS'd. It's fine now.

Would CouchDB be a viable alternative to MongoDB?

CouchDB is designed from the ground up for reliability, so there's no chance you'd have data go missing, even in the case of crashes and system reboots. CouchDB also offers strong consistency guarantees on a single node, so you can use it for things like banking. So... yes.

I remember somewhere in MongoDB documentation that states why you shouldn't use Mongo for sensitive information (i.e. money related) because of the lack of transaction. (No reference, but obviously I read it somewhere because I remembered it.)

Maybe he skipped that part?

http://www.mongodb.org/display/DOCS/Durability+and+Repair

"Of course, some problems require greater durability: we would not recommend using it for a bond trading system."

Indeed we are a trading system :o

Why would records disappear?

If I remember correctly, Mongo batches up writes to disk, and if something goes wrong before that happens, you could lose data. Master/slave setup stuff is the prevention tool of choice.

But I'm not 100% sure. Can anyone more familiar with Mongo internals confirm?

MongoDB does not currently have single-server durability, so a crash can cause loss of data. As you point out, replication is the recommended path to durability. We'll also probably be adding support for single-server durability by 1.8.

That said, the article seems to imply "missing documents" with no crash. That is something I've never seen reported before this post.

Awesome, thanks.

I'm actually in the opposite boat that the author is; I'm trying to find a good excuse to start using Mongo. I need to get better at modeling data without schemas... I don't feel comfortable using it until I get better at the design aspect.

Have you checked out this page: http://www.mongodb.org/display/DOCS/Schema+Design

I think there are also some videos linked from there that should help.

I have not, thanks! I've asked about this topic a few times, and never got a good response. It's not like I was trying hard, though.

Bookmarked. I can't wait to dig into this. Thanks again.

Tried to comment on the post, but so far still "Awaiting Moderation". I have never heard reports of missing documents w/ a production release of MongoDB (or a dev release, for that matter). Would love to help debug whatever issue he saw, but will need more info than that post gives.

Mike, you must have super ears. Let it be a testament to how much you care about MongoDB that I see you very often popping up offering to personally lend a hand helping someone who is having a problem. The handful of production environments I have either used MongoDB in or know of have never had a problem of missing documents when using a production release (or even 1.5).

Thanks! Glad to hear that the effort is appreciated :). We all try pretty hard to make sure everyone has a good experience with MongoDB.

Hey Mike, I'll get in contact with you shortly once I pull some logs. We've been analyzing the data and it seems that where we lose data coincides with some system reboots, we had a few problems with replication a few weeks ago, and had it disabled, so that's likely why we are seeing loss.

Corruption... as I said, doing db.repair often brought the records back, although it did take my application down for quite some time.

are you sure you don't have a HD/controller going bad? From what it sounds like no one has this problem and you haven't given steps to reproduce it.

Rackspace Cloud support is telling me the hardware we are running on checks out 100%.

Reading the documentation about durability and repair[1], it appears data corruption should only ever occur if you had a hardware failure, or the mongodb process crashed or was killed. Was the data corruption you experienced because of any of these reasons? In your experience, how easy is it to repair a node vs switch to a replica node and rsynch. (I'm quite ignorant about MongoDB replication but in my idealised mind, it should be possible to recover a corrupted node entirely automatically and transparently to the client).

[1]http://www.mongodb.org/display/DOCS/Durability+and+Repair

yet again the important question:

Which version was he using? I've never had data loss in a year of production using the "stable/production" version of MongoDB.

Was using 1.4

problem: use SQL or NoSQL?

proposed solution: use SQL or (SQL and NoSQL)

Reply on news.ycombinator.com