How to Do Distributed Locking (2016) | Hacker News Reader

How to Do Distributed Locking (2016)

martin.kleppmann.com

120 pointsappwiz13 comments

In Azure, we solve this with leasing blobs and always renewing, and additionally checking ETags when writing.

That’s cool. S3 is generally fantastic but as far as I know has no similar option to this. I wonder if they’ll ever have a strong consistency mode.

Within AWS, consider DynamoDB with the fantastic DynamoDB Lock Client[1]. It’s a “general purpose distributed locking library built on top of DynamoDB. It supports both coarse-grained and fine-grained locking.”

  [1] https://github.com/awslabs/dynamodb-lock-client

interested to know your approach to long leases lasting more than 60 seconds (if you have this?). do you go for continual renewing (and bailing out if you get too close to the timeout without getting a renewal). or do you go for infinite with your own break logic?

Discussed at the time: https://news.ycombinator.com/item?id=11059738

And the reply from the author of Redis: http://antirez.com/news/101

Seems more like “how not to...”

> Conclusion - I think the Redlock algorithm is a poor choice...

Best book on distributed systems out there! It's one of the books that made go into that area of computer science.

Why is "how to do distributed locking" not "setnx in Redis with timeouts/retries"?

Author says this about setnx:

"If you need locks only on a best-effort basis (as an efficiency optimization, not for correctness), I would recommend sticking with the straightforward single-node locking algorithm for Redis (conditional set-if-not-exists to obtain a lock, atomic delete-if-value-matches to release a lock), and documenting very clearly in your code that the locks are only approximate and may occasionally fail."

I guess I don't understand why setnx locks would fail. Race conditions?

The redis server reboots. Simple as that. Redis is very very rarely run in a transaction mode that guarantees consistency across reboots (and it's rare enough to doubt the code works even when it is set to the necessary mode). If you're using redis in this mode, it's significantly slower than a proper database, so don't do that either.

Is lockService in this example a single machine or a cluster of machines ?

Reply on news.ycombinator.com