Comment by jbellis | Hacker News Reader

Comment by jbellis | Hacker News Reader

Parent Full thread

Of course they are not durable until fsync; that is why having an upper bound on how long that will be is important.

Out of the box Cassandra defaults to an upper bound of 10s. Couchbase defaults to no upper bound at all -- you can lose arbitrary amounts of data on power loss. That's a huge difference.

Since Couchbase does not support a time bound on fsyncs, there are two ways to make a fair comparison: make both systems fsync before acknowledging any write (commitlog_sync: batch in Cassandra and persistTo: master in Couchbase) or give Cassandra an unlimited durability window like Couchbase (durable_writes=false at the keyspace level). Of the two, the former is a lot more reasonable in real world scenarios, but the latter is at least more defensible than apples-to-oranges.

As can Cassandra. It depends on how much data is still in the page cache. I do agree both systems can be configured for immediate durability. However, we went with the default values as most people (and most databases) do not sync on every write. It is too much of a performance cost.

Cassandra cannot lose more than 10s of data because it will slow down writes as necessary to make sure it does not. That's why comparing to a system that allows arbitrarily high data loss is unfair.

People can and do run Cassandra with full durability. When people understand the tradeoff between performance and data loss on power failure, you'd be surprised how often they'll chose real durability. (And by batching concurrent writes into the same fsync, the penalty isn't nearly as high as it would be in a naive implementation.)

Do you place a maximum amount of data in that 10s window? If not, it too is arbitrary. I do think batching concurrent writes is nice.

Of course "limit by time" and "limit by data" are interchangeable; in practice "limit by time" is easier for operators to reason about.

> It is too much of a performance cost.

Its "MongoDB is web-scale" all over again.

Reply on news.ycombinator.com