CouchDB Notice for 1.0.1 - Potential Data Loss | Hacker News Reader

CouchDB Notice for 1.0.1 - Potential Data Loss

couchdb.apache.org

74 pointsamock32 comments

Critical bugs are a question of statistics. As the size of the code-base increases, the probability that there is a critical bug in it quickly approaches one. Who would dare say that millions of lines of core Oracle code do not have at least five obscure bugs that cause data loss in some situations?

There's always going to be some edge case that escaped test coverage or some use case that totally escaped the imaginations of developers in any product. The chances of such an error being discovered is probably a function of the total number of active application instances and some rough measure of complexity in the application's implementation. So yeah, I'd say generally speaking you're right on.

I should make clear that the notice is titled "Notice for 1.0.1" because the 1.0.1 release (due this week) will contain the fix. The bug is confined to 1.0.0.

I hope next time Damien will think twice before insulting other databases for having similar issues:

https://twitter.com/damienkatz/status/15444148375

Good find--I was going to look for it myself.

As I saw the tweet I became uneasy, followed the link and, sure enough, it's my post he's referencing. That's a bit disconcerting, I feel responsible for trashing MongoDB all over the internet. At least my conscience is lighter because everything I wrote was true...

Yes, it was true that the 32bit version of mongodb that is limited to 2gb would lose your data if you injected more than 2gb in it, shame on mongodb...

Jesus, I can't believe people still defend this shit. I don't care if MongoDB can store ten jiggerbytes, it should give an error when you're trying to insert more than that. When the hell did corrupting your data become acceptable for a datastore?

The 32bit version is available for convenience, nobody uses it in prod. You can't complain that a simple testing tool limited to 2gb can't store more than 2gb.

When did I complain it couldn't store more than 2 GB? I complained because it silently corrupted my data.

You used a dev version of a tool that is not meant to be ran in production or for any serious task and then complained it didn't work with your attempt. As this bug is not present in the real prod version of mongodb, it doesn't make sense to criticize mongodb for it. Also next time RTFM before using a tool you don't know much about.

I feel this is a justifiable defense, if you use a pre-release version of anything you should recognize that you are taking a risk.

The 32-bit stable versions have the same behaviour, and the MongoDB guys on IRC did admit that data corruption is not a very elegant way to handle it.

unless you run on EC2 and don't need a large instance (which is where 64 bit starts)

Simple guide to keep your integrity in the software business: don't crap on other peoples products and admit your own mistakes.

I'm confused, are you talking about me or MongoDB? If you're talking about me, I don't think I made a mistake, and if you're talking about MongoDB, the guys on IRC were very civil about it and did say that silently corrupting data was the wrong way to go about it.

It's these apologists who are giving MongoDB a bad name, really, because the guys on IRC were nothing but helpful about it.

Sorry, this should have posted this under the Katz tweet link. My bad.

Oh, that makes sense then, thanks for clarifying...

For pete's sake, for the last time: you were using 1.3.3 which is UNstable and not production ready. Its developers know it's likely to have problems and you proved them right. What you wrote was not "true". You found a bug in beta software.

This instance of CouchDB data corruption is in a production-ready version, aka a stable version. No doubt eventually MongoDB will hit a similar issue and then we'll all calm down and stop poo-pooing all the hard work everyone is putting in building these databases.

For the zillionth time: I moved to 1.4.0 after 1.3.3, and there was data loss on the stable version too. Seriously, what's up with everyone's reading comprehension?

My understanding is that you stopped using MongoDB and started using SQLite. To me, that means that you are probably not a Linux or Mac or Windows user because they all can crash and lose your data. Likewise you are not a user of any office suite because you've likely to lose data from all of them. Likewise, even ISPs can disconnect you half-way through a transaction (say buying something or editing a blog post) and so you will stop using them because of that.

So good luck with historious. I think it's an interesting idea, but by your logic I shouldn't try it in case it crashes and loses my bookmarking data.

I'm pretty sure that my wariness of losing data would mean that I'm... uh... less likely to lose data? Besides, SQLite is amazing.

fwiw, the only error Damien made was failing to review some patches other people wrote. Also, this issues is much less serious than the Mongo problems as it's A) localized and B) fixed. Eg it's a bug not a design decision.

Errors: - not reviewing - poor testing procedures - new code in 1.0.0????

Why your claim that its not as bad as mongo is crazy and self serving: - poor code quality is MUCH worse than a design you don't agree with. how could anyone trust couch? at least mongo is upfront about their design decisions, whether you agree or not - the couch team has gone on and on recently about how unreliable mongo is, and yet here is this... - there were a number of unanswered questions on mikeal's blog - sadly he took them down - probably because he didn't like them. maybe his couch db lost the data and he couldn't figure out why...

Also - really nice job attacking mongo while you guys have a MAJOR bug... classy and proessional...

Another funny thing - how long has 1.0.0 been out? 3 week or so? and this was just found? so what, there are like 10 people using couch in production?

And i'm sure this will be down-voted by the couch fan boys - but this ridiculousness has to be addressed.

> And i'm sure this will be down-voted by the couch fan boys - but this ridiculousness has to be addressed.

Or maybe you'll get down-voted for trying to play psychological games on the voters...

Pure trolling. New user. Jabs at MongoDB aside, CouchDB has acted quickly and professionally to the kind of rare unfortunate critical bugs that can (and do) happen to software projects. This quality of response creates trust, not diminish it. Shit happens. What matters is that you admit the mistake and fix it as quickly and publicly as possible. Props to CouchDB.

The reason the bug took so long to surface is that it is only triggered by an edge case. Most users will not experience issues. After two reports of missing data, the whole team focussed on getting to the bottom of it. This is because we take data integrity seriously.

I don't believe the emergence of a single bug tarnishes an entire codebase and labels it as poor quality. This situation seems like a lapse of judgement in process, which they've fessed to and provided a path to correction for.

Does anyone else find it interesting that the actual underlying bug causing this data loss is related to what seems to amount to a stateful variable? I.e., the "reference to [a] timer is kept in the updater state", and the actual bug being a process which "never cleared the timer reference".

I think this might suggest an additional criteria for extra code review scrutiny in systems (like CouchDB) that are built in functional langages: any code that "feels" stateful or procedural should probably be examined with additional care, as it is the most likely to contain exactly this classic sort of stateful logic error.

So much for relaxing.

Sounds like they were a bit too relaxed on the code reviews.

RavenDB seems like a cool document DB option for .NET / Windows devs - has anyone tried it?

http://www.ravendb.net/

Reply on news.ycombinator.com