Hacker News Reader

Comment by benologist | Hacker News Reader

Why would records disappear?

Tried to comment on the post, but so far still "Awaiting Moderation". I have never heard reports of missing documents w/ a production release of MongoDB (or a dev release, for that matter). Would love to help debug whatever issue he saw, but will need more info than that post gives.

Mike, you must have super ears. Let it be a testament to how much you care about MongoDB that I see you very often popping up offering to personally lend a hand helping someone who is having a problem. The handful of production environments I have either used MongoDB in or know of have never had a problem of missing documents when using a production release (or even 1.5).

Thanks! Glad to hear that the effort is appreciated :). We all try pretty hard to make sure everyone has a good experience with MongoDB.

Hey Mike, I'll get in contact with you shortly once I pull some logs. We've been analyzing the data and it seems that where we lose data coincides with some system reboots, we had a few problems with replication a few weeks ago, and had it disabled, so that's likely why we are seeing loss.

If I remember correctly, Mongo batches up writes to disk, and if something goes wrong before that happens, you could lose data. Master/slave setup stuff is the prevention tool of choice.

But I'm not 100% sure. Can anyone more familiar with Mongo internals confirm?

MongoDB does not currently have single-server durability, so a crash can cause loss of data. As you point out, replication is the recommended path to durability. We'll also probably be adding support for single-server durability by 1.8.

That said, the article seems to imply "missing documents" with no crash. That is something I've never seen reported before this post.

Awesome, thanks.

I'm actually in the opposite boat that the author is; I'm trying to find a good excuse to start using Mongo. I need to get better at modeling data without schemas... I don't feel comfortable using it until I get better at the design aspect.

Have you checked out this page: http://www.mongodb.org/display/DOCS/Schema+Design

I think there are also some videos linked from there that should help.

I have not, thanks! I've asked about this topic a few times, and never got a good response. It's not like I was trying hard, though.

Bookmarked. I can't wait to dig into this. Thanks again.

Corruption... as I said, doing db.repair often brought the records back, although it did take my application down for quite some time.

are you sure you don't have a HD/controller going bad? From what it sounds like no one has this problem and you haven't given steps to reproduce it.

Rackspace Cloud support is telling me the hardware we are running on checks out 100%.

Reading the documentation about durability and repair[1], it appears data corruption should only ever occur if you had a hardware failure, or the mongodb process crashed or was killed. Was the data corruption you experienced because of any of these reasons? In your experience, how easy is it to repair a node vs switch to a replica node and rsynch. (I'm quite ignorant about MongoDB replication but in my idealised mind, it should be possible to recover a corrupted node entirely automatically and transparently to the client).

[1]http://www.mongodb.org/display/DOCS/Durability+and+Repair

Reply on news.ycombinator.com