Comment by antirez | Hacker News Reader

Comment by antirez | Hacker News Reader

I don't agree. For instance Redis Cluster will be consistent (under the limit of physics) and not partition tolerant. But why this requires a network that will never have troubles? Simply when the network will be broken the cluster will not work at all.

What Redis Cluster will guarantee is that you can have M-1 nodes, with M being the number of replicas per "hash slot", that can go down, and/or get partitioned.

So this is a form of "weak" tolerance to partition, where at least a given percentage of the nodes must remain up and able to talk to each other.

But in the practice this is how most networks work. Single computers fail, and Redis Cluster will be still up. Single computers (or up to M-1) can experience networking problems, and Redis will continue to work.

In the unlikely condition that the network is split in two halves the cluster will start replying with an error to the clients.

This means that the sys admins have to design the network so that it is unlikely that there are strange split patterns, like A and B can talk to C that can tolk to D but blablabal... in high performance network with everything well-cabled and without complex routing this should not be a problem, IMHO.

In your first paragraph's example ("Simply when the network will be broken the cluster will not work at all.") you are sacrificing availability.

In the rest of your post, you seem to be sacrificing consistency; one server is down, and thus not receiving any updates from the other servers when data gets updated.

I'm not sure you understood the point of the article, so I'll try to restate it: When part of your system goes down (and it will), you can choose between refusing requests, in which case you sacrifice availability, or serving requests, in which case you sacrifice consistency, since the part of the system which is down cannot be updated when you update data, or cannot be queried in the case of data which is insufficiently replicated. You _cannot_ choose both, since that would require communicating with the downed server.

why do you think my servers are interconnected? I think your conclusions are broken because of many non-always-true assumptions.

In Redis Cluster there is no cluster data communication if not for resharding that only works when the whole cluster is on and is done by the sys administrator when adding a node.

So in normal conditions, a node will either:

1) Accept a query, or

2) Tell the client: no, ask instead 1.2.3.4:6380

All the nodes are connected only to make sure the state of the cluster is up. If there are too much nodes down from the point of view of a single node it will reply to the client with a cluster error.

What I'm sacrificing is only consistency because in every given time there is only a single host that is getting the queries for a given subset of keys.

The exception is in the resharding case that is also fault-tolerant. Or slave election (fault tolerance is obtained via replicas).

As a side note, the clients should cache what node is responsible for a given set of keys, so after some time and when there are no failures/resharding in act, every client will directly ask the right node, making the solution completely horizontally scalable.

Dummy clients will just do always the ask-random-node + retry stage if they are unable to take state.

Edit: there are little fields like this that are totally in the hands of academia. My contribution is from the point of view of a dummy hacker that can't understand complex math but that will try to be much more pragmatic.

"If there are too much nodes down from the point of view of a single node it will reply to the client with a cluster error."

This sacrifices availability. Remember, the cluster doesn't include only the servers; it also includes the clients, since ultimately the point of a database server is to provide the data to the clients upon request.

sure, my tradeoffs are clear, I sacrifice availability in every part of the net where less than M-1 nodes appears to be down in order to win: 1) consistency. 2) latency.

What I did was to stress the tradeoffs that my data model was forcing itself, as Redis handles complex aggregate data and an eventual consistent solutions sucks in this context.

So Redis Cluster users will be a fast scalable consistent solution that will start trowing errors if the network will go down badly, but that will survive if a few nodes will go bad or if there are small network problems affecting a small number of nodes. If this sounds too little available please explain me this:

We have a network with 10 web servers and 10 DB nodes.

The netsplit will split 8 nodes from all the rest, so 10 web servers will be able to talk with 2 nodes.

I wonder how this two nodes will be able to handle all the traffic usually handled by 10 nodes. Netsplit tolerance is a myth if you don't specify very very very well under what conditions.

Re: "In the rest of your post, you seem to be sacrificing consistency; one server is down, and thus not receiving any updates from the other servers when data gets updated."

See: http://news.ycombinator.com/item?id=1768719

Partitions happen, often due to human error. Asymmetric communication scenarios are not exotic at all. It's very common to see them when some link is saturated in one direction. This requires no complex routing, and I've seen it from something as simple as a badly configured backup script.

Also, beware of confusing your local failure detector for a global one. The emergent behavior is a pain when you don't take this into account.

Reply on news.ycombinator.com