50 years later, is two-phase locking the best we can do?

If I have a distributed microservice architecture and I want to keep multiple datastores in synchronization or "consistent" what's the industry best practice?

A few days ago I was trying to solve the inconsistency problem with "settled timestamps" which is a kind of multiversioning idea except that timestamps elapsed with the absence of reported error represent a valid save/commit. Kind of like two phase commit with the second phase being time. The idea is that we watch the clocks of other servers and if they don't update then we know we cannot trust their settled timestamps. (My intent was to allow scaling consistency across many servers, because we don't need to wait for response for every update, we only need to wait for the next timestamp interval)

Here's my Multithreaded multiprocessing Python code to test indeterminancy. 10 threads all send eachother random updates. They also broadcast their own timestamp and the timestamps of their own perspective of the timestamps every other server.

https://replit.com/@Chronological/InconsistencySimulation#ma... (click Run and watch the output, you'll have to wait 10 seconds)

A read in this simulation is the MIN of all timestamps of all servers reported timestamps.

10 seconds into the simulation, we ask every thread for its own perspective of what the counter value is. Sometimes they will all report the same value, a lot of the time they shall be split brained.

I am aware that wall clock timestamps are not suitable for ordering in a distributed system and that logical or vector clocks should be used for ordering.

If you can get the simulation to all report the same number at any point in time, then that would be great :-)

Ordering in distributed systems is significant, as the eventual consistency of the simulation means that some values can arrive late but affect the value, meaning it is not linearizable. Bloomlang tries to solve this.

I'm specifically interested in scaling WITH consistency but I think this is quite difficult.

I'm not familiar with these technologies, but when I was studying databases, SSI was touted as the "better" 2PL on the horizon. I wonder how SSI compares to 2PLSF, and why it wasn't mentioned here?

Often, you can batch the requests and hit the external resource with less concurrency, but larger payloads. If the resource handles well batches, you'll need far less concurrency and locking. For example, if you're working with Postgres, you'll need fewer connections, and you may be able to forego adding PgBouncer, which makes things more complicated.

Granted, batching requests is not something most programming languages are well suited to do. Those that are optimized for high concurrency like Go (channels) and Elixir (processes) can do it well, but for languages that do everything with threads it can be painful.

While it’s not mentioned, I suppose you want to prioritize older transactions in order to prevent long-running transactions from being starved by shorter-running transactions. (If one long transaction conflicts with an average of, say, three short transactions, and on each conflict it’s effectively random who wins, then each long transaction has only a 1/8 chance of winning all three conflicts and being able to commit.)

But preventing starvation only requires older transactions to be prioritized most of the time, not every single time, especially not if the transaction is only slightly older. So some kind of timestamp / cycle counter should work fine, even if there’s skew between threads or other sources of inaccuracy. Ties could be broken by thread ID, or again by having both sides abort.

https://lamport.azurewebsites.net/video/consensus-on-transac...

The issue is if your first locking message gone how do you know it is not the reply message that is lost. For simple one just go ahead and deal with conflict later like GitHub and Dropbox etc. for database good luck. For bank.

Example, typically have 1000 tasks and 10..100 hardware threads.

So, make one ordered list of 1000 tasks, and than make copy of it for each thread, each time randomize order of copy.

Than, each thread will just read it's list and execute task and subscribe to one list implemented as non-blocking multi-thread queue.

In worst case, few threads will do some task repeatedly.

And atomic operations this way will scale up to 1000 times.

All my incoming and outgoing “transactions” are over loosely-coupled API calls.

Is there a point I’m missing?

Sometimes, we find a good solution, and it's the best possible one and we found it early on. We think we can do better, but we can't. A classic example of this is Euclid's fifth axiom. It wasn't proven until the 19th century that this axiom was necessary but everyone from Euclid to Gauss tried to get rid of it. Foolish.