Hacker News Reader

Comment by bitwize | Hacker News Reader

> In essence, it provides a means to absolutely order events, regardless of which distributed node an event originated at.

Einstein said you can't absolutely order events -- and with widely enough distributed systems and small enough time quanta, sooner or later you're going to run into relativistic implications.

Probably not a problem for most applications we currently have to deal with, but one day -- soon enough that we're already giving new protocols names like "Interplanetary File System" -- our databases will spread out among the stars, and how will we handle time and event ordering then?

View on HN

Depends what you mean. Any individual can absolutely order events. It's just they might not agree with someone else's ordering. So as long as you designate an observer, or a couple observers and have a conflict resolution protocol that accounts for whatever it needs to account for (not much if all observers are in the same datacenter, a bit more if they happen to be scattered across galaxies), there's no inherent problem.

Everyone on Earth is in basically the same reference frame. They can easily agree on a standard event ordering.

Even between stars the difference isn't that big, but offsets don't matter at that scale anyway. When it takes a decade to send an email from one system to another, it doesn't matter if their timelines are offset by a week.

You are some several decades in the past. The fact that everyone on Earth is not in the same reference frame affects your everyday life nowadays, because it affects Atomic Time and that in turn affects everything that is based upon Atomic Time.

Since the 1970s, TAI construction has had to compensate for the differences between the physical locations of the atomic clocks in laboratories around the world and an ideal surface of equal gravitational potential around the world.

Nowadays, BIPM and other laboratories routinely talk about general relativistic corrections across the width of the measuring devices. To quote Appendix 2 of the SI Brochure:

> In 2013, the best of these primary standards produces the SI second with a relative standard uncertainty of some parts in 10^16. Note that at such a level of accuracy the effect of the non-uniformity of the gravitational field over the size of the device cannot be ignored. The standard should then be considered in the framework of general relativity in order to provide the proper time at a specified point, for instance a connector.

-- https://www.bipm.org/en/publications/si-brochure/

And despite that, we already have a single unified clock standard that we follow!

Being able to measure the drift at a specific point doesn't mean it's relevant to computers timestamping their calculations. If a computer is ten nanoseconds off, that's basically the same as it being one rack to the left, or having some slack in the cable. There's no real effect.

Just syncing your clock once a day is enough to let you completely ignore the effects of relativity.

For most use cases, you’d just give up serializability and implement an eventually consistent model. For anything that requires something more strict, you just wait. Longer. Processing a check used to take several days. Processing an earth bank check from mars would need to take about half an hour on average. Though that’s perhaps not the best example, because even today we process checks using an eventual consistency model.

Edit: A strict consistency earth > mars transaction would actually take longer, because 30 minutes is just an average round trip, and you’d need more than one.

Note that even with Google Cloud Spanner (last I checked anyway), the distributed consistency model was only available among machines within the same datacenter, as orchestrating transactions across regions would be latency prohibitive. I think you can get cross-region read replicas but that's all. (And I may even be mixing that up with some other cloud database).

So interplanetary, yeah one planet is going to be the central transaction handler and everyone else is going to have to deal with it. So likely things that require transactions, you'll have to have enough cash in the bank in your Mars account if you want quick transactions, and expect wiring money from Earth to take a while.

Coming back to the current day, if you need low-latency global transactions just here on Earth, then that's probably how you'd have to design it. You prime each region with a certain account "limit" of whatever it is you're transacting to use in local transactions, and when that limit gets low you transact some more from your central data store back into your regional account, or something along those lines. It'd be a two-stage thing.

That is pretty much how it’s done today. Global transactions are authorised in real time and then settled periodically. If I remember correctly, Visa does a settlement every 3 or 4 hours, then each of the merchants and other processors in the chain run their own schedule, so it can take a few days for balances to correctly reflect a transaction.

I remember being very impressed by Spanner when I first heard about it, but as you can see in the OP, it does make a lot of consistency guarantees through sheer brute force. It simply throws power/resources at problems which would generally be considered impractical when normally designing a distributed system. An approach which does of course have its limits (not to say it isn’t still impressive, it’s a great system).

Interesting point! I guess the galactic version of UTC would have to include specification of the inertial frame the event occurred in. For the purposes of ordering times could be converted to the time in some standard reference frame, the galactic centre of mass maybe.

Libraries for handling time would need to include functions for converting times between frames.

There is an extension to linearizability that takes General Relativity into account. Paywalled unfortunately: https://link.springer.com/chapter/10.1007/978-3-662-45174-8_... I agree that whatever can be done in an eventually consistent fashion should be. For the rest, at least academia has an answer.

(I work at Cockroach Labs and gave an internal talk on this paper some time ago).

Reply on news.ycombinator.com