FoundationDB vs. Jepsen | Hacker News Reader

FoundationDB vs. Jepsen

foundationdb.com

64 pointsspullara23 comments

Hey guys, I'm the author of the post. I'm happy to answer any questions or comments you may have about it.

This was a good post in general, though you misrepresented Riak by only talking about the situations where allow_mult=false in the Jepsen post. If allowing for siblings, Jepsen and Riak play nice together:

https://pbs.twimg.com/media/BjmZeBYCEAAqCk3.png:large

Thanks for pointing this out. That's totally a mistake on my part. I've updated the table to include the CRDT result, showing Riak's 0% data loss.

Thank you.

It might have been interesting to arrange the partition so that both the master and "cluster controller" went to the minority side. Then a new CC would have had to have been elected before the CC could choose a new master.

I've been playing around with the idea of doing a couple of these posts - for example, I could do one with multiple datacenters. This would be a good scenario too.

How does this compare with http://rethinkdb.com/ ?

RethinkDB doesn't support ACID transactions.

Yep, that's a big difference.

RethinkDB is also a document store, whereas FoundationDB is an ordered K/V store at its core (although transactions / layers allow for richer data modeling through stateless translation "layers").

I don't think there's any data on RethinkDB and Jepsen. Would be interesting to see someone do the work to make that happen.

It is worth noting that an ordered K/V store is a very simple and flexible data structure in terms of the types of operations that can be efficiently layered on top of it. The simplicity gives you guys a chance to do something insanely hard like distributed transactions. The flexibility lets people build a ton of different abstractions on top of it.

has there been one for couchdb yet?

With couchdb you can set up your own couch cluster replication topology:

* master-slave

* a tree

* a ring

* a ring with overlaps

* completely connected graph

Well basically any graph topology you like. Replications can be pull or push. The downside is you are in charge of handling and setting up proxies and deciding how memebership is handled. Whether you have a proxy or not and if so how do client pick were to write etc.

Also CouchDB only offers atomic document transaction. If you need atomic multi-document transactions you have to use a transaction log based on single docs.

But you get continuous replication, a nice browser GUI that helps you debug as you develop, and most of all, a well defined conflict model when there are conflicts in your database.

It would seem to be Jepsen like test doesn't make sense for CDB as it doesn't ship with a default single cluster setup.

There has not to my knowledge. As a CouchDB user I'm definitely interested in the results, but it's not clear to me what cluster configuration to test.

You'll need two pieces to run Jepsen against a database:

1. Code to provision a five cluster database (using Aphyr's Salticid deployment system)

2. Code to do some reads and writes to the database (this is run by Jepsen)

It might be helpful to look through the pull request I submitted to Aphyr for FoundationDB's test, so you can see the pieces: https://github.com/aphyr/jepsen/pull/10

Thanks, that's super helpful. A great writeup, by the way.

Thanks - it was a great way to learn more about the database, and the effect of network partitions in general.

We (Cloudant) ran a Jepsen-style test against our clustered CouchDB implementation (similar to BigCouch). The write-up is at https://cloudant.com/blog/run-dmc-explains-network-partition....

The clustering layer is getting merged into CouchDB and will provide semantics closer to other Dynamo-style systems (e.g. Cassandra, Riak).

What's the significance of the 1 unacknowledged write in the last run?

That means a client submitted a write that was successfully committed, but the client never found out that it was successful. It's more or less inevitable in the face of network partitions.

You don't want the database to report that a transaction was successful before the data is actually committed, or else you lose consistency. But if you wait until after the commit happens, there's a chance that the client might no longer be reachable. There's no way around this aside from waiting an indefinite period of time for the partition to be resolved.

Right. Most of the time you would simply retry the transaction - that's what FoundationDB's automatic transaction retry loop does. The first two tests I ran do that, but the third (foundationdb-append-noretry) does not.

That makes perfect sense. If it is the case, I think it's worth mentioning that the acknowledge write happened when the partition happened. I assumed that it happened when everything was going well, which is odd.

Good idea - I've updated the blog post. Thanks for the feedback!

When a network partition occurs it's possible that transactions be committed but not acknowledged due to the inability to receive the ack.

Reply on news.ycombinator.com