The Unclear CP vs. CA Case in CAP | Hacker News Reader

The Unclear CP vs. CA Case in CAP

blog.thislongrun.com

30 pointsSanderMak13 comments

Normally discussion of CAP results in a heated argument due to the many different interpretations floating around. 6 hours later, not a single comment here on HN or the guy's website.

Not sure if I'm impressed or disappointed considering the author makes some points that I'd never heard before, such as an eventual consistency data store not being CA, CP, or AP.

Probably because there isn't much use to it, as with , such as an eventual consistency data store not being CA, CP, or AP. there is nothing to argue with. He makes the assertion but offers no reasoning to support his claim.

For an existence proof of things he claims doesn't exist consider Cassandra. It is eventually consistent, and you can configure it to be each of those things he says can't happen by tuning the read, write consistency. For CA choose all, CP choose quorum, for AP choose one. QED.

For CA choose all

My understanding is that you can't "choose" CA, because partitions can never be entirely eliminated. If CA was really an option, who wouldn't pick it?

So if Cassandra is actually able to do that, then can you expand a little more on how it accomplishes that?

CA means you're not choosing P, which means your system does not have partition tolerance, and that's exactly what ALL does in Cassandra. If any replica is unavailable, the operation fails, and your application suffers.

This is CP: you have chosen to sacrifice availability by making operations fail when replicas cannot be contacted due to a partition. The concept of "partition tolerance" ends up heavily overlapping with the concept of "availability" to the point where trying to make tradeoffs between the two without sacrificing consistency is essentially nonsensical. As put by Coda Hale: "You Can't Sacrifice Partition Tolerence".

http://codahale.com/you-cant-sacrifice-partition-tolerance/

(Sadly, Coda deferred to someone important after writing this article, someone whom I don't think bothered to think through how much more beautiful Coda's explanation of a "partition" is than the AFAICT impossible-to-discern one that supposedly was being used in the original paper :/. I frankly think the final "update" to the article should be ignored. FWIW, in the first "update", it is pointed out that Eric Brewer actually have a very positive shout-out to Coda's article.)

Ah, I see, I misunderstood Availability, I assumed that returning an error would count.

> I frankly think the final "update" to the article should be ignored. Coda updated his post just because he was wrong on his definition of partition. Stonebraker pointed out: "You simply failover to a replica in a transactionally consistent way. Notably, at least Tandem and Vertica have been doing exactly this for years. Therefore, considering a node failure as a partition results in an obviously inappropriate CAP theorem conclusion."

I said we can build a setup that is not tolerant of network partitions and was CA when it was functioning. I didn't say we could build a system that had no network partitions. P is how well it tolerates network partitions, not whether or not it experiences them.

This post looks at the application as a whole, not only the datastore. But for an eventually consistent big data store, AP is not possible. See http://blog.thislongrun.com/2015/04/cap-availability-high-av...

Thanks for posting this to HN (I'm the post author).

There are two posts: 1) the link you posted. It looks at the application as a whole, not really at the datastore alone. 2) Another one on the big data store themselves, and more generally of the different definitions of 'availability'. It's here http://blog.thislongrun.com/2015/04/cap-availability-high-av...

I will update the posts to make this clearer. As well, it's a series and these posts are the most complex of the series :-). Many points are more detailed in the previous posts.

Interesting article, here are some notes from my reading of it.

First, in the "The same application with an Eventually Consistent store", you don't actually define the semantics of the system, so there is no way to say how it relates to CAP. Eventually consistent stores, like the original Dynamo, are designed to be AP systems, which is why they have things like read-repair. As long as you don't have k failures (i.e., all replicas are down or partitioned), you can always read from a replica, but you may not get consistent data. You can argue here that k-safety is not the same as the A in CAP, but that isn't the generally taken approach (and, if true, it means there's no such thing as an A system, of course, since you can always postulate that ALL networking in the world has simultaneously been cut).

Second, your argument about two-phase commit is combining two different arguments. First, the standard 2PC algorithm does not allow for heuristic commits. That algorithm is a CP algorithm (it is, in fact, referred to as the "unavailable protocol" due to it's inability to make progress in any failure conditions). If you add in heuristic commits, then it becomes a disaster! I can imagine some situations where such a protocol would be useful, but not if you care at all about your data consistency. In that case, why use 2PC at all? Your conclusion is correct in that case: the resulting algorithm is not C or A.

Third, your confusion about CA is that you are trying to apply it to systems, like datacenters full of commodity servers and switches, that don't really fit it. Imagine, instead, a system of two computers connected by multiple, redundant NICs and cables, located next to each other and communicating by sending messages. It isn't hard to believe that you could build a system, like that, in which you could add algorithms that were CA: in other words, if someone cuts the cable, then all bets are off, but as long as those cables remain uncut, the algorithm guarantees consistency and availability. A better example is cache coherency protocols in modern processors, which are incredibly complex distributed algorithms working on CPUs that communicate over tightly integrated channels. Cache coherency protocols need to be CA, of course! If you somehow managed to severe the communication links without destroying the motherboard, you could break the algorithms assumptions, but that wouldn't make it any less of a CA algorithm.

EDIT: Just to be completely clear: the CAP theorem only gives you a very small amount of information about a distributed system. As the author notes, multiple times, just because a system does not meet the "A" in CAP, does not mean it isn't available. It could be 99.99999999% available, and still not meet the A in CAP. The same is true of "C". That's what makes CAP less useful for actually designing distributed systems: many of the choices you have to make come down to what you do when the system is not available, or not consistent.

Apologies, as usual with the CAP theorem, I made an error here. The Gilbert/Lynch paper specifically says that availability only counts for servers that actually receive requests, so if all the network links in the world are cut, you will still be available by this definition. So you can ignore the last two sentences of my first paragraph. That paragraph should say:

First, in the "The same application with an Eventually Consistent store", you don't actually define the semantics of the system, so there is no way to say how it relates to CAP. Eventually consistent stores, like the original Dynamo, are designed to be AP systems, which is why they have things like read-repair. If all your nodes have failed, or all network links have been cut, you are available because there are no requests to which to respond. Otherwise, you can always get a response, and are therefore available, but that response may not be the "latest" response, so it may not be consistent (see Herlihy on Linearizability for the full definition here).

Apologies for adding confusion here.

(cross-porting from the blog)

> Second [...] In that case, why use 2PC at all? Heuristic decisions are in the XA standard since its first version (1991). YMMV, but they are very often used (to say the least) in 2PC production systems. See for example how Mark Little describes them: http://planet.jboss.org/post/2pc_or_3pc. Not really presented as an optional thing. It shows traditional databases are not that 'CP at all cost' when it comes to managing failure.

> Your conclusion is correct [...] the resulting algorithm is not C or A Yeah... I see 2PC as not partition-tolerant as a partition breaks acid-atomicity. Once partition intolerance is accepted, CA fits well: 2PC is consistent & available until there is a partition. Saying '2PC is not consistent and not available but is partition tolerant' is not false technically but it's a much less accurate description.

> Third [...] datacenters full of commodity servers [...] > redundant NICs and cables, located next to each other > [...] It isn't hard to believe that you could build a system, like that, > in which you could add algorithms that were CA

I just totally agree with you here. CAP as a categorization tool is used for all types of distributed systems, but there is a huge difference between an application built on commodity hardware running in a multi-dc config and a 2 nodes system running on redundant hardware in a single rack. Typically, 2PC is historically used on very specific applications: few nodes, a limited number of operations between the nodes, expensive redundant hardware all over the place, limited scaling out needs (if any). Not your typical big data system.

Reply on news.ycombinator.com