Hacker News Reader

Comment by londons_explore | Hacker News Reader

Algorithms that involve 'i did this thing at this time, everyone else should wait for that time to have passed before continuing to ensure correct ordering' (ie. Many database systems) will dramatically slow down.

Systems that use atomic clocks might have been able to do 10,000 transactions per second will suddenly only be able to do 2 transactions per second if they contain a node that doesn't use the same smearing.

View on HN

Those algorithms depend on their reference time being common across all processing nodes - in fact the only system I know that uses wall clock time for inter node ordering is.... Google Spanner, which runs just fine with their smearing approach.

But _all_ of Google's servers will be off by the same offset, which means that they can continue to use their distributed time-based locking mechanism. Generally, if you require that high resolution of time, you don't leave your infra.

Seems a weird way to write an algorithm. Easier to rely on database locking and/or the obvious fact that if you have received some data telling you about an event having happened, then it has happened… Curious if there are any real reports where this caused issues?

If you have not heard of problems with time in distributed systems everywhere, maybe read up instead of guessing? There are literally thousands of papers on the topic, more every year. It is not considered solved.

But where are the reports/blog posts from the thousands of companies using these algorithms that ran into performance problems the last few times cloud providers used time smearing? I’ve done some searching and can’t find any, which suggests this is mostly a theoretical issue.

Yes I could come up with an algorithm where this is a problem, but in most cases I’d favor a solution that doesn’t depend on separate systems having clocks perfectly in sync.

Okay, how many suggest relying on a perfectly synchronized clock versus how many suggest designs which account for slightly desynchronized clocks?

If you have a distributed system that requires strict time synchronization why do you care if Google’s time is wrong? Surely if you need that level of synchronization you have your own implementation with gps clocks.

If you don’t, then you already have to deal with standard drift with ntp anyways so Googles time smear is the least of your problems

One does not always have the luxury of systems operated entirely within one organization.

If you don't have the luxury of managing all your systems within one organization, then time smear is the least of your problems. Day-to-day ntp drift alone will be a bigger problem.

Refer to “Practical uses of synchronized clocks in distributed systems” for some info on algorithms like this.

https://dl.acm.org/doi/10.1145/112600.112601

Google spanner uses stuff like this with a clock called TrueTime. I don’t know how leap seconds / smearing is handled in that system.

Reply on news.ycombinator.com