You Can’t Sacrifice Partition Tolerance (2010) | Hacker News Reader

You Can’t Sacrifice Partition Tolerance (2010)

codahale.com

43 pointsxrorre21 comments

Maybe I'm missing something, but it seems like this is basically just glossing over this part:

> Some systems cannot be partitioned. Single-node systems (e.g., a monolithic Oracle server with no replication) are incapable of experiencing a network partition. But practically speaking these are rare;

I'm not sure how the writer came to the conclusion that these systems are rare. The website I'm working on now has a single monolithic Postgres server, and the majority of systems I've worked on connected to single monolithic datastores, so at the very least these systems exist. These aren't particularly interesting examples, but when talking theoretically like with the CAP theorem, you don't get to just ignore the rare or uninteresting cases. These seem like relatively common examples of choosing CA.

Of course, when you try to shard such databases, you're going to run into serious problems.

I guess the claim I can credit to the author is that CA really really really means no partition tolerance. Even one network partition puts you in CP or AP territory. But I don't think that means CA stores don't exist.

It may be that I'm just not understanding things correctly, though; the CAP paper is definitely a challenging read for me.

You skipped the second part of that. Unless your web application is running on the database server then you are no longer running single node, your web app and db server can become partitioned.

Ah, okay, I didn't understand that next part correctly the first time. So that does account for why the author says such systems are rare.

But that doesn't really help me understand why CA can't exist. There are still systems that simply don't have network partitions. SQLite on the same phone as the app using it? As I said, it doesn't seem like we ignore the rare or uninteresting cases.

The CAP theorem is fundamentally about distributed systems. The name is on the tin. It's not ignoring things, you're trying to pick an argument via examples outside the explicitly defined scope.

Clustered CA systems can exist and they corrupt data (maybe silently) when a partition happens. In most discussions it is taken for granted that people don't tolerate data corruption.

Single-node "CA" systems are "available" under CAP's definition but not available under the common sense definition. A single hardware failure will take out the entire system (but in theory everything will be in perfect order after you restore from backups).

A "clustered CA" system that corrupts data sounds like a AP system.

During a partition an AP system just returns stale data but a CA system may completely crash or lose all data.

Even if your web app and database are on the same server, you may experience partitions between the client and server.

You can, but it's vastly less likely than networking partitions assuming your DB is operating as expected.

If you read the next sentence he says

> But practically speaking these are rare; add remote clients to the monolithic Oracle server and you get a distributed system which can experience a network partition (e.g., the Oracle server becomes unavailable).

The theory in question addresses distributed database systems. A remote client connected to a monolithic database is not a distributed database system. If there is a network partition between the client and the server then you don't get C or A. You get nothing.

The P refers to partitions between nodes of the database system. Not clients.

So that point in the article doesn't seem to make sense.

At this point we're edging into a semantic argument, but I would consider client-server architecture to be a type of distributed system. For the record, wikipedia lists "Client server architecture" on their page of distributed systems too: https://en.wikipedia.org/wiki/Distributed_computing#Architec...

> You get nothing.

You don't get nothing, you get inconsistency that could require even human intervention to fix. For example, if client lost connection right before the database was able to respond that his commit was successful. Client then retries and you get a typical double billing problem.

That's because monolithic database with a remote client is in fact a distributed system and strong consistency in a distributed system requires proper distributed algorithms and protocols, where clients are nodes too. You cannot guaranty consistency otherwise (to be precise, you will need clients to be able to wait indefinitely to guaranty strong total consistency, but if clients are used by humans they will not do that and the only consistency you can guaranty for a human is eventual consistency).

Also, per the article, a single node system WITHOUT REPLICATION. That would hopefully be quite rare, as it means you have a single point of failure, with no automated recovery plan.

But as soon as you have replication...you have distribution, and what happens when a partition occurs, where the master can no longer talk to the replica(s)? That's where CAP applies; the system -will- make a decision about what to do, and in doing so will trade availability for consistency, or consistency for availability, for any given read/write scenario.

> The website I'm working on now has a single monolithic Postgres server, and the majority of systems I've worked on connected to single monolithic datastores, so at the very least these systems exist.

These systems do not guaranty neither consistency, nor availability during network partitions.

The implication derived from a trivial probability argument is misleading. Failures are correlated, so 1-(1-P)^n is an upper bound, firstly, but secondly and much more importantly, the value being computed is the wrong value. Failure of a node is not partition. To non-trivially partition a cluster of N nodes requires (N-2) failures, when the cluster is fully connected, for example (as e.g. on any bus network). Handling trival partition is, well, trivial, in some important senses.

It's worth keeping in mind that viewing a system as CP or AP usually isn't nuanced enough: https://martin.kleppmann.com/2015/05/11/please-stop-calling-...

Liked everything up to this part;

"When it comes to designing or evaluating distributed systems, then, I think we should focus less on which two of the three Virtues we like most and more on what compromises a system makes as things go bad."

It seemed like the entire point was that really there are only 2 Virtues (Consistency, Availability) and given a partition you will chose one or the other. The only database that doesn't have network partitions also isn't distributed.

> and given a partition you will chose one or the other

There is no sacrificing consistency though. It's about whether you use CRDTs or something to keep the system working when network partition happens or you don't and let the system stop working.

You can (and a shocking number of distributed database systems in widespread usage do) fail to guarantee either one.

The original CAP paper was intended as a high-level discussion item for students. Brewer has since emphasized that it is more of an academic approach than applicable to real-world distributed databases.

Other computer scientists have either modified or narrowed it to be more useful.

So it's fun to read the original CAP paper, but it's less useful than you would expect. When somebody asks me about CP vs. CA, I realize they in fact don't anything at all about distributed databases.

Reply on news.ycombinator.com