Does MongoDB 1.3.x (dev) silently lose your data? | Hacker News Reader

Does MongoDB 1.3.x (dev) silently lose your data?

korokithakis.net

48 pointsbravura35 comments

MongoDB VERY CLEARLY warns you about the limitations of the 32-bit install when you spin it up. When I invoke mongod on my 32-bit development box, I get a nice big message:

  ** NOTE: when using MongoDB 32 bit, you are limited to about 2 gigabytes of data
  **       see http://blog.mongodb.org/post/137788967/32-bit-limitations for more

If you manage to miss that, it's your own fault. Combined with the fact that he was using an unstable development branch, he has absolutely zero room whatsoever to complain. He hit a documented limit and got less-than-friendly behavior on an unstable build of the software. Going nuclear over it is just silly, and makes him look inattentive and/or naive.

That seems like a brutal limitation done for non-obvious reasons ... I mean, wtf?

> makes him look inattentive and/or naive

Oh I wish you gave me more options. This is not a proper ad hominem.

It's a brutal limitation if you're expecting MongoDB to be "MySQL, except with more awesome because it's new and shiny!", sure, but once you read the blog post, it's quite apparent why - memory-mapped files are blisteringly fast, and raw speed at the cost of other features (like relational constraints, transactions, and single-server durability) is a part of the MongoDB setup. If those are acceptable sacrifices, then great, enjoy a wicked fast data store. If they're not, then it's not the tool for you, no matter how much the cool kids chat it up.

Spot on. I came here to say just this. Dev branch or not, there is no way he could have missed this. I don't know how you can better advertise a limitation than this. Maybe he didn't realize he was on a 32-bit box. Maybe he didn't pay attention to how much data he would be using.

If you want to use Mongo for something serious, make sure you're on a 64-bit box. Memory-mapped file I/O is an important part of what makes Mongo so fast and attractive, but it has obvious limitations on a 32-bit box.

Title is misleading, he was using the unstable dev branch. There's no evidence here that the stable branch does this.

His excuse at the bottom of his post is way lame. It's very clear what is and isn't stable on MongoDB's main download page:

http://www.mongodb.org/display/DOCS/Downloads

Flagged, but a title change would be acceptable, too.

I have updated the title in response to this criticism

Thanks! I unflagged it.

Come to think of it, the article is still a good cautionary tale. What it really teaches is:

a) do your research

b) use the production version if you're using code in production, unless you have a really good reason not to and have some safeguards in place

The fact that it was "unstable" branch doesn't justify data loss or corruption. It can be used as an excuse to explain why server crashed, but not why data was lost. This is big no-no in my book.

Also, if the 500k documents limit on 32-bit architectures is true then it's just sad.

so one class of bugs is OK, but another is not? scratches head.

Have you ever done software development?

I never said that bugs are "OK", but yeah... Some bugs are acceptable and some are not. Data loss is never acceptable, especially not with databases.

And yes, I did my share of software development...

Except in this case, it's not a bug, it's a design limitation. The MongoDB folks make no bones about it - the software is very limited on 32-bit systems. It warns you about it. It tells you to use 64-bit servers. If you insist on using the tool beyond the bounds for which it was designed to be used, that's your problem, as the developer.

Is it a "bug" if you redline your car's engine and cause it to throw a rod? Or is that simply you pushing the machine past its design limits, despite the warnings not to do so?

Data loss / corruption is design limitation?

Yes. MongoDB doesn't ensure a write to disk before returning on an insert operation (it is what is called "eventually consistent") in order to maintain high write throughput. Because it uses memory-mapped files, it can't address anything larger than a 32-bit pointer on a 32-bit system, and so you would execute an insert, and when the DB eventually tries to write it to disk, it would fail due to lack of resources. Because the write is asynchronous, the client doesn't realize that the data has not been serialized to disk, and you end up with effective data loss. Fixing this would require either a) not using memory-mapped files, or b) ensuring full durability on writes, both of which would massively slow down the data store as a whole.

Bugs can be of varying severity. In some environments a crash bug (especially when using a dev branch) is tolerable and can be recovered from quickly and easily. In some systems this is par for the course and there are processes whose job it is to restart crashed processes all day long. I can't think of too many places where a data loss bug in an application whose sole purpose is to store and provide access to data would be even remotely acceptable.

Not only that, but he fails to grok the reason for there being a limit with the database size on 32-bit systems. It's directly a result of MongoDB using memory mapped file IO (http://en.wikipedia.org/wiki/Memory-mapped_file) which gets better performance, but on a 32-bit system you're constrained to the amount of memory addressable by a 32-bit process. So rather than do somewhat hacky (and complex) code they opted to be forward looking and just leave the limitation in place. Thankfully, with a 64-bit OS the upper limit goes up to 256 terabytes.

I use MongoDB, collecting just under 200k documents a day for months and I haven't lost any data yet so far that we've found. Sqlite looks pretty speedy though - may try going back to something like that for smaller jobs.

The one thing I found funny is that deleting documents from a database leaves the space allocated and you have to manually run the repair command to reclaim the space, but that requires 100% additional space because it creates a new db and copies the data there.... little bit of a problem since we were already a 99% for the device :-| other than that - been really happy with it.

That's not terribly uncommon. PostgreSQL also does not reclaim space from deleted rows until you vacuum. Many hash table implementations also only mark items deleted rather than actually removing them.

Although - for further clarification, PostgreSQL doesn't require 2x the total amount of disk to reclaim/reuse space under normal circumstances.

As of version 8.1 (out since 2005), VACUUMing is an automatic function inside the server. (autovacuum)

SQLite is really nice for small projects, but it rapidly degrades once it hits large amounts of data. It's wonderfully useful for many things, but managing large datasets is not one of those things.

Er, wow. I don't get the impression he has any idea what's going on.

The title is incredibly misleading, especially given that this is a ~2 month old post.

As of this writing, 1.4.x is the stable branch, with 1.5.x as the unstable.

I should note that I've been running MongoDB in Production since last August. Development, deployment and go Live occurred on a pre-GA 0.9.x version and I never encountered ANY issues. With almost a year of uptime I've had no data loss or anything else.

At the same time I understood I was using an unstable version and was careful to also understand what was going on under the covers.

So here's the REALLY important thing you should understand if you're using MongoDB because it probably has to do with his data "loss":

Data operations in Mongo, viz. insert/update are asynchronous - from both a API client and a disk-persistence standpoint. One of the things that gives you the speed boost you see from MongoDB is this concept.

When your language driver sends new rows, it does NOT by default wait around to see if Mongo saved it correctly. It sends it off, makes sure MongoDB got the data and returns to you. You can force it to wait, and you can ask it for the lastError - but normally you don't wait for a "I got it in correctly" answer.

Additionally, once MongoDB receives it, it goes into memory... it writes to disk lazily, rather than immediately (if you sent 1000 increment commands between the last disk write and the next, it would batch them into a single increment, etc).

Yes, these things can lead to data "loss" - or the appearance thereof. If you're sending in a bad update or insert statement and not checking for errors, data will "disappear" - by which I mean it was bad data and never accepted for write.

Using MongoDB was a mistake to begin with even if he didn't select the wrong branch to use. His project isn't about databases, it's about machine learning. For his requirements, he just needs a simple datastore and it sounds like SQLite would have worked fine.

He procrastinated by trying out cool new software (a classic symptom of a project that is not going properly) and lost a lot of time because of the mistaken manner in which he did it.

He should get back on track by focusing on the real meat of his project and use the simplest bitbucket he can find for it.

Yes, the fact that MongoDB was losing his data was all his fault because he chose MongoDB. Sound logic.

That's not what I said.

Well, the problem was the unstable branch. The general case larger problem was that he wasn't paying attention to what the hell he was doing.

Oh yeah, hi BJ :)

No project is "about databases" [1]: they about something else, and databases are a tool use to achieve that end.

Besides, whether he was procrastinating or not is irrelevant to his complaint.

[1] ok, some are.

If you are doing document processing, a document database is actually the right choice.

I disagree with your comment. I do machine learning and NLP, and I prefer mongodb as a data-store to RDBMs. SQLite would be overkill. MongoDB is a more convenient and simple choice than an RDBM.

Using a schema-less document database is natural in NLP, where you might add columns to your fields on the fly, e.g. you are experimenting with a new preprocessing step and want to cache the result.

I'm curious what you mean by "overkill." SQLite has really low startup costs and is easy to use from almost any language. The code you have to use to query it is not verbose. Typically when I see the word "overkill," I expect that there is some downside for using an approach that is too powerful. But I can't think of a downside to using SQLite in this context.

SQLite requires schema definition (CREATE TABLE, ALTER, etc). Thats a bit overkill sometimes when you just want to store a JSON object. Mongo makes it much easier to just start pushing data into collections without having to worry about the tables and columns being defined.

hasnt for me, so far ... been running three heavily written to mongo installs for more than a month...

I hope you have backups and have tested restoring them.

Agreed, but that has nothing to do with the use of MongoDB. Any data that "can't be lost" needs to be backed up.

It should be noted that this post is over two months old.

Wahwahwah. This guy sounds like he has never used open source software before. I have a complex mongodb install in production with no problems. Every time I "bundle install" it breaks something because everything is running edge and the APIs are changing all the time. If you are not competent to deal with that then maybe you shouldn't be playing with shiny new toys.

Reply on news.ycombinator.com