10gen's response to MongoDB's slams
readwriteweb.com
readwriteweb.com
Unfortunately, it was less about 10gen or MongoDB, and more about the efficacy of Hacker News as a forum for disseminating information. Plug in any controversial technology and the article sounds about the same: Al Gore on Global Warming vs a report claiming to debunk it, Microsoft on Silverlight vs a report claiming that it's no longer supported, James Randi on his $1,000,000 challenge vs someone claiming that their version of ESP really is real.
"... the reaction on HN is heartening. Though it's disappointing it made it to the front page at all, it also seems that the bulk of the audience at HN took it with the grain of salt it deserved."
I'm unimpressed with this assessment. HN participants were quick to identify the key issues of provenance while simultaneously picking apart the claims. Regardless of whether this was a hoax -- I personally think it was an exaggerated rant of someone who was seriously frustrated -- isn't this how peer review is supposed to work? Someone makes a claim. The claim and the claimant are assessed. Conclusions are drawn. There's nothing disappointing about it.
In my opinion, HN worked efficiently and quickly to debunk or validate the claims. The article needed to reach the front page in order to achieve a critical mass of participation in order to discover a consensus which seems to be that many of the claims were overblown or based on out-of-date information, but some were real. This conclusion was validated by 10gen's president Max Schireson,
"... rather than deflecting the entire thing, Schireson and Horowitz were fairly candid about MongoDB's shortcomings. Of the nine sections, Schireson says that "some are definitely valid, some we haven't heard or seen.""
Rather than feeling disappointed, the newsworthy aspect of this episode is how well the HN community and moderation system worked in this case, and how reasonable 10gen was in their response. Many companies (e.g. British Petroleum on the Deep Horizon disaster) will continue to spin the news even when there is video evidence.
It would have been nice if ReadWriteWeb focused on that rather than attempting to paint a negative picture of community sourced news.
There's a lot disappointing about it. HN, like it or not, has gotten to a point where many people use it for tech news and don't read comments. When a story makes it to the top of HN, it gets tweeted to an audience of about 40K people - who generally don't see the comments at all. (links go straight to stories, not to the HN site.)
So there's a huge potential for misinformation when a piece like this makes it to the front page at all.
The peer review worked, but in my opinion a better system wouldn't even put this in front of a large enough audience to matter. It was an anonymous pastebin - it's not like this was debunking something on CNet or RWW that would have been widely seen anyway.
We vote on things that are interesting to discuss, not to indicate there's any validity to the article. If non-discussion-participants are relying on the voting mechanism to provide "accurate tech news", they've only got themselves to blame for the "huge potential for misinformation".
People should be upvoting articles based on quality, not headline-bait or baseless rants (no data of bugs, no logs, etc) or rants where the person is oblivious (locks are in the docs).
Unfortunately, I can't downvote links. Like other sites, I can only ignore it for so long before it's drowned in noise (lack of interesting quality links).
I don't really understand your position or complaint - are you suggesting that the anonymous nature of the source is sufficient to disqualify it outright? Who determines 'baselessness', isn't that a consensus process? Are you commenting on a general feeling that HN promotes a lower number of quality links then "in the earlier days"?
Yeah, but by what measure of "quality"?
I don't hang around here with the intention of curating non-participating readers news feeds, I keep coming back for the interestingness of the discussions that result from submitted links. The true or falseness of the linked article is way less important (to me) than the opportunity for interesting comments.
('course, maybe _I'm_ doing it wrong...)
They are nice for getting hints on what's going on, but it's stupid to plainly believe stuff - like everywhere.
It's the same with cyber mobbing and posting random stuff about or even impersonating another person. If people wouldn't simply believe all the nonsense, be it FUD, advertisement or whatever a lot of things would be better.
This also seems to be a problem in the scientific field. Things sadly aren't transparent all the time. There are secrets and press releases that are made for investors. On the other hand there appear to be fewer peer reviews. Maybe because it's hard to make money doing so (especially when there is a lack of transparency in first place). If you want to discredit you competition it's far more effective to create FUD than it is to find (probably hidden) errors. Investors, consumers and even the media is unlikely to be interested into actual, scientific data that weaken or even disprove claims.
Generally, the claimant is irrelevant in peer review. That's a hard standard to hold up though.
The submission did not have the data (logs, bug reports, etc).
So it's pretty much easy to disprove the rant as nothing more than a rant.
So I looked through Mongo's change logs and one of the first things I saw for version 2.0.0 is a "huge" issue: "reIndex() on secondary drops all indexes" https://jira.mongodb.org/browse/SERVER-3866
This sounds like a bad bug for a database to have. And in this article 10gen president says that data loss hasn't been a problem since 1.8, which seems to contradict their own change log....
That doesn't excuse a large bug like this from existing, but it doesn't contradict his assertion.
If MongoDB makes it possible for you to do something unique, that no other proven solution would allow, it might be worth the risk.
If you used MongoDB because you heard it's the latest hip technology, and your company suffers, you made a stupid mistake that you'll hopefully learn from.
for massive user generated content where big numbers of statistics come into play, and noone would die if you lose their hard-composed 140 char tweet, use transaction- and lock-free, easily-sharded, rarely flushed to disk stores.
10gen definitely handled it perfectly.
Specifically, it would have been nice if they outlined the known issues surrounding compaction and how much of an impact not running compaction will have after weeks or months.
Essentially, almost every reasonable use case absolutely requires either taking a regular outage for compaction or using a 3-node replica set to do a rotate-compaction dance. This is an EXTREMELY frustrating thing to run into and surprises like this that pop up after deployment are unlikely to earn them any allies.
10gen has been publicly talking about working on online, incremental compaction for over a year now. Riak, CouchDB, Cassandra, Redis (AOF), and HBase all have incremental, online compaction.
Is this something HN can fix?
When I vote something up (as I did with the article mentioned), I'm thinking "this seems interesting", not "this seems authoritative and well sourced". The idea that someone might go through the effort of writing something like this as a "prank" didn't even enter my mind. Perhaps it might help for there to be some short-circuit mechanism for publicly flagging false stories which are found to be blatantly fabricated (or is it a rare enough occurrence that it doesn't matter)?
Shouldn't it be? I think HN was perfectly fine in this regard. There are a lot of companies that aren't 10gen that would take this, go to a court, and try and dig up the person who wrote the article to sue them for libel, whether the article is true or not.
Just because in this case it was a hyperbolistic rant, doesn't mean there won't be a situation in the future where there's valid points being raised against a company that might retaliate against the author.
And as has been noted, the first comments on the article were counterpoints to it.
Please show me the cases of a company taking someone to court for libel for making technical complaints such as this.
It happens all the time on blogs and in mailing lists where people sound off like this about products (rightly or wrongly) and I have not heard of a single, solitary case of a person being taken to court for libel by a tech company.
By the way - the odds are that any company that tried that would lose, not only in court but also in public opinion.
Other companies may not handle it as well as 10gen - but I've yet to see a company try to take someone to court.
Soon after I saw the rant, I blogged about how I work around some MongoDB issues. Also, anyone who uses MongoDB without carefully reading the documentation (apparently like the ranter) is going to have problems.
What matters is what kind of mission the software successfully accomplished and how.