A distributed locking library built on top of Cloud Spanner and TrueTime
github.com
github.com
If you start having global spanner replicas, don't you start getting into read safe times? According to the GCP docs, a given node may not know if it has the latest data, as global replicas could diverge by up to 10 seconds ( assuming I'm reading the docs right).
I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use.
Chubby: https://research.google/pubs/pub27897/
Spanner read write docs: https://cloud.google.com/spanner/docs/whitepapers/life-of-re...
Chubby is a lot like etcd except without the hindsight. Distributed locking is pretty complicated and if I remember correctly there was a great podcast from Google's Kube podcast about distributed locking that goes into a lot of this. [0]
If you are going to do any distributed locking I strongly recommend listening to it. Has a lot of pain points discussed.
[0] - https://kubernetespodcast.com/episode/123-leader-election/
I think one of the issues with this is that its far easier to use chubby incorrectly than to use it correctly. Like, you can use chubby to build really, really cool and powerful tools (https://www.usenix.org/system/files/conference/osdi16/osdi16...), but I wouldn't be surprised if most users misused or overused it, and that would likely be magnified on a public version.
On the other hand Spanner must have a perfectly fine distributed lock manager on its own. It seems odd to implement a lock manager on another lock manager (which just happens to be a database too) just because you can't directly access it. I'm sure that must cause unnecessary overhead.
It's a bit like those odd nesting cases that happen in networking, like PPPoE over Ethernet. Maybe that's the eventual fate of computing as it evolves, more and more layers everywhere, that are more and more redundant but still exist for some odd reason, with low level facilities being reinvented.