Hacker News Reader

Comment by room271 | Hacker News Reader

Unfortunately, this guy doesn't understand the CAP theorem at all.

Once you are distributed, P is not an optional. Rather, in the case of network failure, consistency or availability is what suffers. A system cannot be both CA and distributed.

So, for example, Elasticsearch is not a 'CA' solution despite the diagram in the article, but is actually closer to PC (although, in practice it is far more subtle than even that as it is not perfectly consistent, and configuration options allow for some trade-offs between availability and consistency in the case of communication errors.

View on HN

I can confirm. I design distributed databases; this "explanation" is one of the worst I have ever seen (which is saying something).

Could you please point us to the "best"?

I've commented elsewhere in this thread with a list of better articles and papers. If you've only got time to read one, I'd probably read http://research.microsoft.com/apps/pubs/default.aspx?id=1926.... In fact, if you do nothing else, just look at figure 2 from that paper.

Thanks to mjb for the good list. To it I'll add: http://markburgess.org/blog_cap.html

I think I also might write one up myself.

In case anyone is still watching this thread--my version:

http://blog.foundationdb.com/minimal-explanation-of-the-cap-...

A system can maintain C and A despite a partition as long as no requests need to cross the partition boundary. In practice this isn't something you can count on, and CA systems tend to have absolutely horrible failure modes when reality intrudes, but - unless we redefine "partition" to mean a partition that actually gets in your way - they're not strictly impossible.

Longer version (approx. four years old) here:

http://pl.atyp.us/wordpress/index.php/2010/10/when-partition...

If you've got no coordination then distributed systems are much easier. That's a good design goal, for sure, and distributed database designers spend a lot of time thinking about where they actually need coordination. On the other hand, a lot of very interesting applications - like high-availability ACID - require coordination.

Something always bugs me with CAP theorem explanations, and I am really confused by all that: usually, I read (or understand from those articles) that a system is partition tolerant when "the system is available and consistent while one server is down (=one server is out of reach)".

But, what I think is actually true is: Partition tolerant means that nodes still respond to requests while some other nodes are out of reach (one node being one or more instances of the DBMS in one cluster).

Am I right?

So, If that is true, having a RDBMS with master<=>master replication between two sites is Available and Partition tolerant, albeit inconsistent (the replication is not possible).

In the case of a master<=>master replication as a load-balancing/failover solution, it's about availability and consistency, but we should not even call that "distributed".

Still right?

I think, if you take the view espoused in http://codahale.com/you-cant-sacrifice-partition-tolerance/ , that P can be explained as the "the system is predictable across a partition."

Geography and whether it's a "failover solution" doesn't change that the system is distributed - anything with more than one node is fundamentally distributed, and I'd argue that even two processes communicating on one instance can be considered as a distributed system as well.

The system can choose which kind of predictable it wants to be: whether it sacrifices consistency in favor of being available (accepting reads and writes without knowing that reads are fresh or writes linearizable) or availability in favor of consistency (rejecting reads and writes that can't be confirmed).

In the case of "master<=>master replication" as it's configured in most relational databases, I think the system tries to be AP: conflicted writes or stale reads are possible because barring special options like 2-phase commit, the replicas generally lag each other (as anyone who's tried to reconcile a MySQL split-brain situation knows, this can be a tremendous pain).

It makes sense, thank you!

Also, he seems to think that C vs A is decided when one chooses a data store, when in fact it can be done at the granularity of individual queries. For example, Cassandra allows reads and writes to one node, the majority of nodes (quorum), or to all nodes.

There's some debate on the Elasticsearch forums around CA vs. CP - with the overall consensus in favor of CA. Elasticsearch doesn't perform well across networks so in practice, most deployments have multiple nodes in a single datacenter - CA.

Thanks for your feedback though, I have some posts queued up on more of the subtleties in the model, this was just meant to be an introductory post.

Elasticsearch actually sacrifices both consistency and availability during partitions, even when inside a single datacenter (trust me, they happen in production).

If you configure ES to prioritize consistency somewhat (minimum_master_nodes), it prevents writes during a partition—but there's at least one "split" partition scenario where even minimum_master_nodes doesn't prevent inconsistent writes. If you configure ES to prioritize availability during a partition, it isn't consistent. Remember, ES doesn't claim to be consistent and doesn't even use any sort of consensus algorithm.

I feel like my original comment was a bit harsh so thanks for replying.

Re: the debate about CA vs CP, I've seen this too, but I think it reflects confusion rather than genuine options. The 'P' of CAP takes effect whenever you have two or more processes which clients can communicate with. So even if you are running within the same datacenter when using ES, provided clients can communicate with more than one ES node, CAP still applies.

Reply on news.ycombinator.com