Hacker News Reader

Comment by lurker458 | Hacker News Reader

Is the cost of doing these remote reads worth the savings in this case ? After all to be correct, they need to run in the same transaction at the sending side and need to be persisted at the receiving side. Also, consider the effort if this gets scaled up to 10 cases (each node would have to do 9 remote reads)

View on HN

They don't need to be persisted at the received side, but yes, there could be 9 remote reads if each of 10 shards are running code that may result in state-based aborts. Usually, however, only a subset of the shards run such code.

But remember, 2PC requires 4 messages per worker (so 40 messages total in your example).

True, but it's O(n^2) complexity instead of O(n) complexity (1000 shards would lead to 1M reads instead of 4k)

If they're not persisted at the receiving side, how does it handle a crash before committing on the receiving node ? Keeping previous versions indefinitely on sending nodes to permit requesting the old values doesn't work, so this introduces a time bound on how long a crashed node has to recover (granted, this could still mean weeks)

This requires a longer response that I have time write now. Please check back tomorrow night ...

magicalhippo asked a similar question both with regard to complexity and garbage collection. I think it is best to combine these threads, so please see my response there ... https://news.ycombinator.com/item?id=19003212

I don't understand your point about only a subset of shards running code that may abort. Since we are talking about transactions, if there is even a single shard S that may abort (based on its state), then all other shards must also abort when S aborts. Which would imply that all other shards must also read the state of S before they commit. Am I missing something?

Reply on news.ycombinator.com