Knossos: Redis and Linearizability
aphyr.com
aphyr.com
I keep thinking Antirez is being forced into these problems by community pressure, though. There seems to be a lot of effort going into trying to make Redis some kind of super-featured distributed mothership of a database when we already know that it's not going to be possible.
Antirez and the community have put in an incredible amount of effort to create a datastore that is exceedingly good at what it does, probably best-in-class. To start asking for a distributed system with more guarantees than it was fundamentally designed for is unreasonable. The WAIT command, like every other command in Redis, was written to serve one purpose and do it well in bounded time. I expect that purpose was chosen keeping in mind what was possible without completely demolishing the current Redis architecture. If we can build a reliably distributed system using it, that's fine, but I don't think we ought to be demanding it this way.
the model tested here was again nothing planned to be implemented, just a model to show that WAIT is just a low-level primitive, and the real behavior of the system is determined by all the parts of the system, not WAIT per se.
Redis Cluster is not going to be a strongly consistent system, I'm not going to be tricked into designing something is not good for the Redis Community in my opinion. However Redis Cluster tries with heuristics to put bounds on how much a replica can diverge, and to pick the history that makes more sense when a slave is promoted.
Btw, about the Aphyr post, 13 days ago I wrote in the Redis mailing list the following message in reply to him:
---
About non linearizability, perhaps it does not apply to the case where a strong coordinator exists, but in the general case one issue is that when we read, we can't just read because a stale master could reply with stale data, breaking linearizability. There is a trick to force the read to be acknowledged that could work:
MULTI
INCR somecounter
GET data
EXEC
WAIT <n>
The fact you can't read from the master without care is a very obvious and well understood problem in distributed systems. Even a DS newbie like me was able to spot that in 5 minutes during the mailing list thread :-)
This is personal opinion, of course, but I think Redis would be better served by not striving to make formal guarantees of any sort. Instead it could continue to provide primitives while staying fast and stable. Heuristics should be acceptable and gradually improved as more approaches become feasible. This seems to be the approach being taken right now - I'm hoping you and everyone else can let that be.