Why Flowdock migrated from Cassandra to MongoDB | Hacker News Reader

Why Flowdock migrated from Cassandra to MongoDB

blog.nodeta.fi

43 pointsmutru11 comments

Clearly articles like this are designed to promote the service rather have any substantive "hacker" news. That said, the front page looks compelling except for one thing -- NO PRICING. It doesn't matter if it's still in beta. You are asking me to commit to a completely new workflow -- it's not worth my time unless I have some idea how much you are going to hit me for once it comes out of beta. $9/mo? $99/mo? I mean c'mon, give me something...

Note that flowdock was on the obsolete Cassandra 0.5; the little they described of their problems suggested that upgrading to the 0.6 series would solve the worst of them.

Actually, we were running Cassandra 0.6.1. I'm a bit sorry that we didn't try to upgrade to 0.6.3, because it could have fixed some of the problems.

I have noticed a disturbing trend used in the defense of many NoSQL products --> When faced with criticism point out that there is version n+1 now available and they should have used that, it would have fixed everything, etc.

We've seen this, ironically, in reference to many MongoDB horror stories, so it's interesting that in this case it's the case where someone is moving to MongoDB.

To be fair, many of these products are in their infancy so yes in general the fix may be in version n+1. I don't think that's a defense of NoSQL products it's a common thing with any young product.

Anyone who chooses an early product would hopefully understand that they will be experiencing issues and will be upgrading the product many times as issues are found and fixed.

Cassandra has a lot more installations pushing hard on it in 0.6 than it did in 0.5, many of whom have worked with me and other committers to resolve their issues, so it's not just paint-by-numbers handwaving on my part.

Here are some of the changelog entries from 0.6.2 that are relevant to high-volume sites:

- improve default JVM GC options (CASSANDRA-1014)

- better defaults for flush sorter + writer executor queue sizes (CASSANDRA-1100)

- added Collections.shuffle(splits) before returning the splits in ColumnFamilyInputFormat (CASSANDRA-1096)

- StreamingService.StreamDestinations wouldn't empty itself after streaming finished (CASSANDRA-1076)

- Add option to turn off Hinted Handoff (CASSANDRA-894)

- split CommitLogExecutorService into BatchCommitLogExecutorService and PeriodicCommitLogExecutorService (CASSANDRA-1014)

from 0.6.3:

- add ability to lower compaction priority (CASSANDRA-1181)

... and 0.6.4:

- extend option to lower compaction priority to hinted handoff as well (CASSANDRA-1260)

- pre-emptively drop requests that cannot be processed within RPCTimeout (CASSANDRA-685)

If that's so much greek to you, I'll be doing explanations of these over on the Riptano blog in the next few days, like the one I did for 0.6.1 here: http://riptano.com/blog/annotated-changelog-cassandra-061

(sorry for excessive spacing. how do you get HN to format bullets?)

In you article, you mentioned 'Dots are not allowed in BSON document keys. Typically it might not be a problem, but we had to work around it in our data migration.'

Do you have more information?

In MongoDB this is not possible:

> db.test.insert({"foo.bar": 1}); Mon Jul 26 14:24:39 JS Error: uncaught exception: can't have . in field names [foo.bar]

After all that's only natural, because the dot notation is supported in queries.

For example, consider storing an e-mail in MongoDB:

{ content: "<p>HTML mail. <img src="cid:1234.0" alt="from attachment"/></p>", attachments: { "1234.0": { some details about the attachment } } }

This is not legal, but in this case it's of course easy to work around (have attachments in an array and have the CID as a value instead of a key).

Thanks, this is interesting.

Quick question: is the service meant to work in the latest DEV release of Chrome? I just get a blank screen: http://cl.ly/92a3f95975927920868b

Indeed, latest development version of Chrome doesn't work. We found and fixed the bug earlier today, the fix will be deployed later this week.

Reply on news.ycombinator.com