The Simple Magic of Consistent Hashing (2011)

paperplanes.de

31 pointstosh3 comments

Odd, the article claims to be about consistent hashing, but spends all its words on plain-old hash quality topics.

> If that partitions just so happens to carry the largest object by far in you whole cluster, that’s something even consistent hashing can’t solve. It only cares for keys.

Isn't that the one of the problems solved by foreign keys?

not really? its a question about the underlying space. but yes, consistent hashing does have an answer for this, which is if you can agree on a metric for business, and you have a a way of communicating occupancy (i.e. through a membership protocol), then you can adjust your number relative number of replicas to move some load.

within reason, if you have an object that takes up half the storage on a node, then you can end up moving it around quite a bit for getting down to hosting only the replica that has that object, and even then all the occupants of that bucket might be more than you can hold.

but CH is such a lovely notion, because it gives you those kinds of tools. like the fact that it not just provides an object-server mapping, but actually an ordered list of servers.

Reply on news.ycombinator.com