Hacker News Reader

Comment by kd913 | Hacker News Reader

Why does Facebook need a precision time server?

To improve the performance of their distributed systems by imposing stronger assumptions than full asynchronicity on their environment, I expect. (See also Google’s Spanner.)

Of course we go inspired by the Google Spanner project and what they have published over a decade ago. Hats off to the Google Spanner team

Sure, I didn’t intend to make any deep point here; I don’t think I have any deep points to make in this particular area, honestly :) I just wanted to drop a set of easily searchable terms for motivation that might not have been made explicit in your writeup (probably because it’s quite tangential to it—metrology and distributed systems are both fascinating but don’t really share a lot of ideas).

It seems Google has a better time determination method than the initial TrueTime used in Spanner now. https://www.usenix.org/system/files/osdi20-li_yuliang.pdf

There are probably a few reasons. One that I can imagine is that they use a distributed database somewhere that benefits from reliably tight wall clocks (in the style of Cloud Spanner or Cockroach).

Cockroach is much less dependent on highly accurate clocks. That was basically the core goal. Spanner like, without expensive clock infrastructure. Though it sounds like CockroachDB does better with at least reasonably well synced clocks.

https://www.cockroachlabs.com/blog/living-without-atomic-clo...

True, I'm not disagreeing with you. However, it's worth nothing that Cockroach DB's consistency model assumes consensus in 180ms, and Spanner assumes similar SLOs at 6ms. Arguably, one could say that the reason why an open-source spanner-like DB doesn't exist in the marketplace with comparable performance is because these types of cards are proprietary and close-source (i.e Google's TrueTime). To your point, Cockroach is much less dependent on highly accurate clocks, but if these PCI-e cards became a common hardware commodity Cockroach can become even faster.

Now imagine a world where distributed DB technologies get so good that the trade-offs between consistency and availability is enough to warrant the use of a distributed DB for mid-sized projects. Right now, these DBs are seen as muscle cars with high cost and high maintenance only for BIG projects. A card like this might be the game-changer.

Would be nice. Though it still requires an antenna that can see the sky, and the cabling.

From the article: "More accurate time keeping enables more advanced infrastructure management across our data centers, as well as faster performance of distributed databases."

That seems rather ambiguous and doesn't really explain why having more time precision results in faster performance or 'advanced infrastructure management'.

I can understand why a stock broker or trading firm requires PTP to enable precise date-stamps for auditing/validating trades.

I don't see how having a time granularity on the order of picoseconds is needed for a data center.

In distributed databases that offer transaction semantics, they need timestamps to order transactions that take place. A tighter synchronization of clocks mean they can execute transactions faster because they can reduce the amount of time they wait (based on the potential clock drift between machines in the data center)

Wouldn’t it be better to put the effort towards designing a system that doesn’t need precise timekeeping?

See https://cloud.google.com/spanner/docs/true-time-external-con...

It should explain how clock uncertainty relates to throughput of causally-related transactions.

When you have collisions, the timestamp might be the thing that decides which update came first. The higher the precision of the time stamp, the faster you can run stuff.

If you know that your clocks are exactly the same in New York and Japan you can assume that if an account creation timestamp in Japan is before the account creation time in New York for the same username that the account should go to the Japan user and the New York user will be told that he cannot use that username. Previously the way to handle that problem was to have all account creation be done on one server which will block the table while it is doing writes. That doesn't scale when you need to insert millions of rows a second.

PTP is required in a number of 5G implementations. Facebook's 5G core (Magma - https://github.com/magma/magma) utilizes it quite a bit.

Reply on news.ycombinator.com