Hacker News Reader

Comment by bbss | Hacker News Reader

Very similar to how BigTable[1] works under the hood which was built ~20 years ago.

[1] https://static.googleusercontent.com/media/research.google.c...

The load shifting part is similar to the way BigTable splits, merges, and assigns tablets. But the rest of it is not related, because BigTable does not try to offer mutation consistency across replicas. If you write to one replica of a BigTable, your mutation may be read at some other replica, after an undefined delay. Applications that need stronger consistency features must layer their own replication scheme atop BigTable (such as Megastore).

What this post is describing for replication seems more comparable to Spanner.

I don't understand this comment. Bigtable requires that each tablet is only assigned to one tablet server at a time, enforced in Chubby. There's no risk of inconsistent reads. Of course this means that there can be downtime when a tablet server goes down, until a replacement tablet server is ready to serve requests.

Right, the contrast I was trying to draw was between what they depict, where multiple nodes are holding a replica of the tablet and performing synchronous replication between themselves, and what BigTable would do, which is to have the entire table copied elsewhere, with mutation log shipping. What they are doing is more analogous to how Spanner would do replication.

Unless you're doing multi-cluster replication, there is no log shipping in BigTable: the data replication within a cluster is taken care of by the underlying filesystems.

Single-cluster BigTable is strongly consistent.

There can be consistency issue if you run multi-cluster BigTable, because multi-cluster replication is asynchronous.

Reply on news.ycombinator.com