Hacker News Reader

Comment by loxias | Hacker News Reader

Two observations/questions, the first probably naive:

* It's not that hard to get your own "world class" time server, for under a thousand. A Rb standard slaved to a GPSDO is gonna be so accurate and stable, and use that to drive a SBC that supports IEEE1588, where you run your NTP and PTP server. Oh, but I guess that box, while inexpensive, isn't in Amazons DC, so doesn't help you.

* PTP's absence in the Amazon Time Sync Service article is quite conspicuous!

View on HN

Amazon time sync likely also requires extremely accurate clocks on the system board as well to prevent there from being too much drift.

You don’t even need a tcxo. If you don’t care about holdover, connecting the pps pin from a $5 GPS module to an $15 SBC will give you a time source that is significantly better than anything you can serve over NTP to your clients. NTP is always gonna be the weak link for a service like this.

I was referring to the amazon time sync service which I imagine has pretty tight timing requirements (similar to spanner), not NTP.

Also, how exactly would GPS work at scale for 10,000s of servers within 1000s of racks within a DC?

"Amazon provides the Amazon Time Sync Service, which is accessible from all EC2 instances, and is also used by other AWS services. This service uses a fleet of satellite-connected and atomic reference clocks in each AWS Region to deliver accurate current time readings of the Coordinated Universal Time (UTC) global standard through Network Time Protocol (NTP). The Amazon Time Sync Service automatically smooths any leap seconds that are added to UTC.

"The Amazon Time Sync Service is available through NTP"

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time...

As I understand it, “amazon time sync service” is just a bunch of NTP servers run by Amazon. Their ClockBound library calculates error bounds the same way you would if you were using a different NTP server: |Local Offset| + Root Dispersion + (Root Delay / 2)

>how exactly would GPS work at scale for 10,000s of servers

GPS module (+ GPSDO + Rb clock optional) -> SBC running NTP server -> machines running NTP client

I’m not sure what you meant by “clocks on the system board” but if you’re talking about the system clock on the NTP server or the NTP client, it really doesn’t matter. Any old crystal on the NTP server is accurate enough to stay locked to a PPS input. The clients are limited by NTP so their clocks don’t really matter either.

I see. This different than what I thought it was. If it is indeed a simple NTP server, this makes sense.

By using PTP in the data center LAN to distribute the GPS time locally.

https://en.m.wikipedia.org/wiki/Precision_Time_Protocol

That's when you move from NTP to PTP, which in a LAN provides sub microsecond accuracy.

https://en.m.wikipedia.org/wiki/Precision_Time_Protocol

With that underneath you need just one gps clock reciever for the whole LAN, and PTP capable Ethernet adapter and switches throughout, of course. Which most are, these days.

Do you have experience with this? Playing around with PTP has been on my TODO list for quite a while, but it seems to be quite a hassle.

At home, I already have a GPS module with its PPS pin connected to an Pi running an NTP server. In my data center, I use clock.sjc.he.net and that's about as good as NTP gets. Certainly good enough for one-way latency measurements.

Are there any SBCs with hardware timestamping and a PPS input? I've heard mixed reviews of the BeagleBone Black as a PTP server. Then again, even if I get that working I have no way to run PTP in the data center since there's no GPS reception in the cabinets.

It _is_ a hassle, and you should have a need for that level of synchronicity. We did it in our team for an in-vehicle network, with the vehicle GPS reciever as the network PTP time master and we figured how to access the ethernet PTP register both from userspace and from the kernel. Kernel for the system clock, user space to avoid latencies from kernel Userland interaction in some high precision real time task. It was a major effort and it required low level hackery way above my personal head.

Them providing PTP would be really interesting

To be clear, what I meant was "the fact that they're advertising whiz bang time synchronization, but then not using the correct protocol, makes me question the quality of the whole enterprise".

Even on local networks, NTP can only get you so close. If you set up chrony just so, in ideal conditions, I've gotten hundreds of microseconds (more commonly ~500-1000us). But combined with PTP, you can get sub-microsecond accuracy.

When Microsoft did that kind of thing we called it “embrace and extend.”

That sounds different. Embrace and extend means implementing a protocol and then adding specific non-spec features. For example, postgres 'embraces' SQL and 'extends' it with specific features - in the future postgres could 'extinguish' SQL by breaking from the spec, forcing an ecosystem split.

Thank you. I was in the process of writing this exact comment but kept tripping over my own words and eventually gave up.

Microsoft already does offer PTP on Azure, and nobody calls it EEE.

https://docs.microsoft.com/en-us/azure/virtual-machines/linu...

Reply on news.ycombinator.com