Hacker News Reader

Comment by hodgesrm | Hacker News Reader

For this to work as easily as portrayed in the article it would imply that distributed consensus must be possible in the presence of failures. But such consensus is unsolvable [0] so it seems that this approach is just moving the problem to another place.

It's difficult from reading this article to understand exactly what that place is. I would guess that it must at least in part involve limitation on the behavior of applications to eliminate category 1 application generated aborts. If so what do those applications look like?

[0] https://en.wikipedia.org/wiki/Consensus_(computer_science)

View on HN

Right. Talking about "Calvin", he says:

> Nonetheless, it was able to bring the processing of that transaction to completion without having to abort it. It accomplished this via restarting the transaction from the same original input.

I may be misunderstanding, but if you have transactions recorded in such a way that any worker can replay from any point in time and reach the correct database state, then the transaction log must absorb all of the complexity. The problem has moved, but not changed.

The big difference is that it's a much easier problem to solve in the context of a log (see Paxos, Raft, ZAB, et al), where the consensus membership is more or less fixed, as opposed to having to manage it among an adhoc membership set.

No. Each process is independently checking the value of a non-changing snapshot of the state of the input data and we therefore do not run into the impossibility result.

So then whatever process maintains the snapshots of input data must absorb all of the complexity eliminated from the workers, no?

Multi-versioned systems can read from a snapshot at negligible complexity (just read the correct version).

And is the snapshot system also distributed and highly available? If so, wouldn't it take on the challenges of committing changes?

There is no separate snapshot system. Each partition just maintains multiple versions of all data it stores locally.

Do I understand then correctly that the:

      temp = Do_Remote_Read(Y)
      if (temp > 0)
         X = 42

Is actually

      temp = Do_Remote_Read(Y_i)
      if (temp > 0)
         X = 42

where Y_i is the value of Y in the version i that matches the version for which the coordinator created the code?

Otherwise Y can be something else? And isn't then there a need for a synchronization with the keeper of Y, if Y_i is still not available?

Yes, but it is possible to architect the system such that Y_i is available.

> it is possible to architect the system such that Y_i is available.

I'd really like to know how? How can we be sure that the keeper of Y is ready (or even online) at the Read moment?

(As you see I'm not familiar with the systems you worked on but I'd really like to understand the context in which you present the solution).

I think I misunderstood what you were saying. By "available" I mean that the version exists at the time of the read. As far as available wrt failure, this is no different than any other system. If you don't synchronously replicate data, you can cannot process transactions over data that is offline (and still guarantee ACID). The transaction cannot proceed whether or not you are using 2PC.

That's exactly where I got stuck on the example. (Saved me editing my comment. Thanks!)

> (just read the correct version)

Famous last words!

Reply on news.ycombinator.com