Virtual Consensus in Delos
blog.acolyer.org
blog.acolyer.org
I read (very quickly) the paper, then the article.
First of all, all the new names for things that are basically shards are not helping. Then all of this is very domain limited to logs, with only append and truncate.
Yes, you can build dbs on append-only, and well, good for logs, but the title read like more on the lines of playing against raft. The abstract pointed out it's a db.
Is this mainly about consensus? Probably not, I don't see any reference or benchmarks with raft in the middle. Barely any about Zookeper, which is not known to be really cutting edge. It's 10X the speed of... what they had before. which is? Besides, this is about a database build on rocksdb, did we manage to reduce comparison between databases to "10x than something else"?
So you have "conventional shared log api", and each shard can implement things in its own way. Good idea, but apparently the API is too conventional to discuss it, but novel enough to have a paper about it?
So I guess this is one of those frustrating things where you have to learn a whole distributed system to understand some new algorithm or abstraction.
I'm not saying this does not have merit, I'm sure it has, and I'll be happier when I understand it. I'm just pointing out that (at least to me) this is really frustrating to read.
It's a distributed database built on sharded append/truncate logs. They managed to abstract some operations to test new algorithms and implementations. The title makes it looks like all of this is about consensus, which (to me) it looks like only a small part. There is no benchmark except against older versions of itself, with only one "10x" measure. The database is not open source.