Comment by hinkley | Hacker News Reader

Comment by hinkley | Hacker News Reader

Parent Full thread

My first long-term job involved collecting telemetry data from equipment and displaying it.

The display was technically challenging, but in the meaty way that developers often relish. The clock skew, however, was not.

For liability purposes it was sometimes necessary to know if Event A happens before Event B, which means you have to normalize all of the events across time zones and then correct for drift too.

That experience and a bunch of others (including statistics classes, and studying the Java Memory Model, which other languages have borrowed or stolen) have left me with a lingering doubt about how we record distributed activities.

I really kind of feel like we [all] need a new data model [like the one they mention here] where dependent events are recorded in that way. I don't know exactly what that would look like, but I think it would help a great deal in consensus situations where you have to resolve a conflict, or even just for displaying a sequence of events in proper order.

It feels like we keep trying to get the exact nanosecond when something happens, but the only thing I ever see humans use that information for is to reverse engineer a sequence of events that resulted in a peculiar state in the system.

[edit: tie-in to article]

I think you might find this seminal paper interesting and relevant: https://amturing.acm.org/p558-lamport.pdf

From the abstract:

> The concept of one event happening before another in a distributed system is examined, and is shown to define a partial ordering of the events. A distributed algorithm is given for synchronizing a system of logical clocks which can be used to totally order the events. The use of the total ordering is illustrated with a method for solving synchronization problems. The algorithm is then specializedfor synchronizing physical clocks, and a bound is derived on how far out of synchrony the clocks can become.

I will read that, thank you.

You know, it seems like most of my quality of life improvements over the past 20 years has been due to my peers and I finally acting on much older information. The future is here, it's just unevenly distributed.

It is only infrequently I encounter something that still feels properly new under any kind of scrutiny, instead of revealing itself to be a refinement of something that already was known. Off the top of my head, I can think of escape analysis, Burrows Wheeler transform, and the object ownership semantics in Rust. I'll throw Raft on there since the joke is that only 12 people understood Paxos.

Sounds a lot like logical clocks https://en.wikipedia.org/wiki/Logical_clock, especially something like a vector clock https://en.wikipedia.org/wiki/Vector_clock.

>It feels like we keep trying to get the exact nanosecond when something happens, but the only thing I ever see humans use that information for is to reverse engineer a sequence of events that resulted in a peculiar state in the system.

Lamport clocks do exist and can provide a partial or total order of events. Not sure if that would've helped with your problem but such algorithms do exist. But it feels like they are rarely used (subjective feeling).

It should be noted that the total ordering that Lamport clocks provide is fairly artificial (in fact it's best to say that there is no natural total ordering of events in a distributed system).

Agreed, though you do see them a lot in CRDTs: https://en.wikipedia.org/wiki/Conflict-free_replicated_data_.... Just about every non-trivial CRDT has something akin to a logical clock embedded inside it.

Reply on news.ycombinator.com