Hacker News Reader

Comment by jbellis | Hacker News Reader

Given that Couchbase is very aware of my criticism of the last time Thumbtack benchmarked C* vs Couchbase, it's significant that no mention is made of efforts to have both systems do durable writes. It looks like they made the same mistake a second time: http://www.datastax.com/dev/blog/how-not-to-benchmark-cassan...

(Update: the Couchbase pdf says that they used "the fastest consistency and durability modes available for the particular database" as cover for benchmarking different things... but even that wasn't done right; no mention is made of setting durable_writes=false in Cassandra.)

View on HN

Well, writes are not durable until fsync. That's true for MongoDB, Cassandra and Couchbase Server. That being said, Cassandra demonstrated great write latency. The issue was read latency.

Of course they are not durable until fsync; that is why having an upper bound on how long that will be is important.

Out of the box Cassandra defaults to an upper bound of 10s. Couchbase defaults to no upper bound at all -- you can lose arbitrary amounts of data on power loss. That's a huge difference.

Since Couchbase does not support a time bound on fsyncs, there are two ways to make a fair comparison: make both systems fsync before acknowledging any write (commitlog_sync: batch in Cassandra and persistTo: master in Couchbase) or give Cassandra an unlimited durability window like Couchbase (durable_writes=false at the keyspace level). Of the two, the former is a lot more reasonable in real world scenarios, but the latter is at least more defensible than apples-to-oranges.

As can Cassandra. It depends on how much data is still in the page cache. I do agree both systems can be configured for immediate durability. However, we went with the default values as most people (and most databases) do not sync on every write. It is too much of a performance cost.

Cassandra cannot lose more than 10s of data because it will slow down writes as necessary to make sure it does not. That's why comparing to a system that allows arbitrarily high data loss is unfair.

People can and do run Cassandra with full durability. When people understand the tradeoff between performance and data loss on power failure, you'd be surprised how often they'll chose real durability. (And by batching concurrent writes into the same fsync, the penalty isn't nearly as high as it would be in a naive implementation.)

Do you place a maximum amount of data in that 10s window? If not, it too is arbitrary. I do think batching concurrent writes is nice.

Of course "limit by time" and "limit by data" are interchangeable; in practice "limit by time" is easier for operators to reason about.

> It is too much of a performance cost.

Its "MongoDB is web-scale" all over again.

Reply on news.ycombinator.com