NTPsec is not quite a full rewrite | Hacker News Reader

NTPsec is not quite a full rewrite

esr.ibiblio.org

72 pointswtbob36 comments

    * strcpy, strncpy, strcat:  Use strlcpy and strlcat instead.
    * sprintf, vsprintf: use snprintf and vsnprintf instead.
    * In scanf and friends, the %s format without length limit is banned.
    * strtok: use strtok_r() or unroll this into the obvious loop.
    * gets: Use fgets instead. 
    * gmtime(), localtime(), asctime(), ctime(): use the reentrant *_r variants.
    * tmpnam() - use mkstemp() or tmpfile() instead.
    * dirname() - the Linux version is re-entrant but this property is not portable.

And so, instead of competent systems code of 1995 vintage, we get ESR code of 1998 vintage.

Presumably, someone else, somewhere else, is solving the problem that needs to be solved, which is to get an NTP implementation that is simple but complete enough for the 98% use case deployed based either on thoroughly modern C code (which looks nothing like a heap of strlcats and snprintfs) or a safer language, like Rust or Go.

Maybe that's ntimed, maybe it's something else. It's not this, whatever the hell this is going to turn out to be.

strlcpy and c99/POSIX-2001 didn't really become widely used until the early 2000s. I disagree that the difference in "vintage" between early-2000s and today's software makes the real difference - what really matters is the quality. And it's not like this project doesn't use DVCS or static analysis. It even uses a modern alternative build system!

It's very hard to quantify or even make specific rules about quality. These rules are just the basics. The strlcpy rule in particular is simplified, there are times when you need to use strncpy: when the source string might not be terminated (but openbsd guys say they use memcpy in that case ... anyway, it takes some thought).

2004 vintage C code is the sweet spot for portable systems software right now IMHO. But the real difference in quality can't be identified by any simplistic rule or attribute.

So, were I to seek examples of such "thoroughly modern C code" in order to educate myself, do you have some suggestions where to look?

In particular, would you recommend to do string handling with some string library (e.g. glib has one?), or just use the mem*() functions and manually keep track of the lengths, or what?

Check out the recent AGL story on BoringSSL for an idea of what a conversion from 90s C code to modern C code looks like.

Yes: do string handling with high-level string abstractions. No, don't try to freelance it with the mem-() functions.

I really like the description of ntimed, but I'm worried there's been no blog updates or code uploaded to GitHub in months. https://github.com/bsdphk/Ntimed

To be fair, we know about the progress on NTPsec by ESR's posts, but there isn't a public repository.

systemd-timedated looks promising for replacing ntpd for most client use cases.

Interesting, this and phk's ntimed are both sponsored by the Linux Foundation. I guess it makes sense on some level to have concurrent projects that tackle the same problem from different directions.

Here's ESR's comment on ntimed (http://esr.ibiblio.org/?p=6863&cpage=1#comment-1635375):

esr on 2015-10-09 at 09:22:05 said:

>Eric, what do you think of ntimed and their goal of replacing ntpd?

PHK is a bright guy with many clues. I respect him a lot, and I say this despite the fact that he’s hostile to me because he thinks my open-source advocacy has encouraged crappy programming. I have not looked closely at ntimed but I have little doubt it is high-quality work, probably as good as anything I am capable of producing, possibly better.

That said, I think ntimed is going nowhere. A clean-sheet design was the right thing from a technical point of view, but it’s not what the time-service userbase wants. Large time-service users are intensely conservative and risk-averse; what they want, and what NTPsec will deliver, is not radical innovation but a cleaned-up and hardened version of what they know.

The only active GitHub ntimed fork is for the Amiga/AROS version, and phk's GitHub visible work as of late seems to be mostly on Varnish Cache.

>> I have not looked closely at ntimed but I have little doubt it is high-quality work, probably as good as anything I am capable of producing, possibly better.

Ok, that is just precious. =) One is a kernel hacker and time-nuts regular with one of the world's most precise clocks in his home, and the other recently wrote:

> There’s a longstanding legend that only Dave Mills ever really understood the Byzantine time-synchronization algorithms at NTP’s heart, but I used to be a mathematician and I think I already get most of it outside of a few arcana about statistical filtering of noisy signals.

From a purely technical standpoint, I know who I'd bet my money on.

By the way, phk recently mentioned that he was getting back into ntimed, so fingers crossed that we get two safe and correct NTP alternatives in the observable future.

https://twitter.com/allanjude/status/635635386832715777

While he said as of August 23rd that he's agin working on ntimed, that's not reflected in GitHub updates: https://github.com/bsdphk/Ntimed

And it bears mentioning that ESR has been working on GPSD for more than a decade (a daemon to extract accurate time from GPS receivers), and he sure sounds math literate to my not entirely untutored ears in various discussions over the years.

Given the well known curse of total rewrites, which I've experienced myself, I too know who I'd bet my money on.

> "he sure sounds math literate"

This is the standard ESR parlor trick. Eric took several courses in math and philosophy at the University of Pennsylvania.

http://poynder.blogspot.com/2006/03/interview-with-eric-raym... "As a freshman at the University of Pennsylvania Raymond was immediately marked out as a potential math prodigy. Having found school insufficiently stretching for his above average talents, however, he lacked the necessary discipline or emotional maturity to cope with the demands of an undergraduate course, and after suffering a "math burnout" left without a degree."

I'd put my money on phk. Every. Time.

Saying that GPSD is "a daemon to extract accurate time from GPS receivers" is a little misleading. GPSd primary function is to make "data on the location/course/velocity of the sensors available to...the host computer." It also has functionality to share data with ntpd but timekeeping is certainly not the primary purpose.

> ESR has been working on GPSD for more than a decade

Timekeeping and location calculation is very easy to do clean sheet. That is, until you start to hit corner cases and exceptions and the realities of physical real world inputs like jitter and offset and dispersion (and those are just the grit that ntpq tells you about) and all the crud humans made up like leap seconds and calendrical calculations.

My money's on the guy who's been blowing the grit out of the machine.

Oh, certainly. But from what I've followed of this project, that part of nntpd is, while complicated, healthy and neither needs serious modification nor is slated for much more than maybe adding tests. It's the huge pile of cruft that accumulated around the core that's getting worked on, or simply removed.

...precious.

Sure, ESR is kind of silly, but I don't understand the animus here on HN. ISTM he isn't making anything worse. Somebody wanted NTP "Classic" improved, and they decided to pay ESR to do it. It's not as though his patches or those of his collaborators would have been incorporated into ntimed anyway. Maybe some people think that work like this should only be done by a PhD with the approval of a committee of PhDs. ESR has been pretty consistent in all his voluminous writings, that he does not agree with that development philosophy.

However I sympathize with everyone in this thread who runs ntpdc on his Lisp machine.

Look, all things considered I like that we have someone like esr. He's an important figure in the history of software engineering. Whatever my opinion of his software output might be, that doesn't affect his impact a bit.

But the man's writing - and that statement in particular - is just so pompous. And here of all cases we have two opposites colliding. One is a world-class expert on the topic in question, NTP, and the other is a world-class self promoter. So when esr says that Kamp might `possibly produce something better` than him while working on an ntpd successor, I can't help but smile. =)

> one of the world's most precise clocks in his home

Seriously? More precise than the clocks in national labs that contribute to the derivation of UTC? (USNO, NRL, NIST, NPL, NRC, etc) I don't think that is a claim phk has ever made and I am not sure why you are making it for him.

phk has a Stanford Research Systems PRS10 Rubidium Frequency Standard (0 MHz rubidium oscillator)<http://www.thinksrs.com/products/PRS10.htm> at his home. Serial # 005597. The short-term stability of this unit is about 5 × 10-12

This particular PRS10 is GPS referenced _and_ drives one of his GPS referenced NTP servers.

He also has (or had) an array of 10 Motorola Oncore UT+ or M12+T GPS receivers sitting on a row 20cm apart, mounted on a shed roof.

UTC(USNO) is provided by the USNO Reference System #2 (Master Clock #2), which is a Datum Inc. hydrogen maser http://tycho.usno.navy.mil/maser.html

The frequency stability "sigma" of MC #2 has been < 2 x 10-15 during the last year.

NIST-F1, the nation's primary time and frequency standard, is a cesium fountain atomic clock developed at the NIST laboratories. The uncertainty of NIST-F1 is continually improving. In 2000 the uncertainty was about 1 x 10-15, but as of January 2013, the uncertainty has been reduced to about 3 x 10-16

So, no, not "more precise" than the clocks you list, but how many of us have a Rubidium standard in our home?

(And when compared to the large number of clocks in the world, yes, I think that the PRS10 is "one of the most precise".)

So more precise than all of the clocks in the world once you exclude "the most precise clocks in the world"? Yes, MC#2 is a hydrogen maser clock. But you failed to mention that it is driven by an ensemble of 41 clocks, 4 rubidium fountains, 19 hydrogen masers, and 18 5071 cesiums. And if we are talking about NIST-F1, don't forget F1's new friend, NIST-F2, that is three times as accurate and soon enough the Italians will have there very own version IT-CsF2. And then there are the lattice clocks from JILA...

Welcome, fellow time geek.

"One of the most..." doesn't mean "the most". It means there is a group of "the most", which can include a large raw number of objects, but this number is still a tiny fraction of the overall number of things, and his is in that group. It could be the last one in that group, but still in that group. Saying "The united states is one of the most populous nations on earth" doesn't mean it's more populous than China or India, simply that it is up in that rarified group of "mosts".

My bad, I wasn't completely precise. The claim is he has one of the most precise NTP clocks, which was relevant to this discussion.

https://queue.acm.org/detail.cfm?id=2716278 (Ctrl-F 'precise')

Let's look at what phk had to say about esr's advocacy.

https://queue.acm.org/detail.cfm?id=2349257

Pull quote:

"That is the sorry reality of the bazaar Raymond praised in his book: a pile of old festering hacks, endlessly copied and pasted by a clueless generation of IT "professionals" who wouldn't recognize sound IT architecture if you hit them over the head with it. It is hard to believe today, but under this embarrassing mess lies the ruins of the beautiful cathedral of Unix, deservedly famous for its simplicity of design, its economy of features, and its elegance of execution. (Sic transit gloria mundi, etc.)"

I regularly drop that link when I see people talk about FOSS's many eyeballs like they inherently mean something. That said, I recently got to read Cathedral and Bazaar. In that, he made good points esp about parallel debugging. I think the truth is somewhere between their two claims. Plan on trying to figure out the specifics in the future.

I lean more toward PHK given FOSS model has never produced a high assurance system while proprietary and core team models have done many:

https://www.schneier.com/blog/archives/2014/04/reverse_heart...

Recently I've been redefining the situation to try to find a model that balances the benefits of open-source against the ability to pay a competent team to ensure it's a cathedral. Start of that is here:

https://www.schneier.com/blog/archives/2014/05/friday_squid_...

From what I could find ESR's work is funded by ICEI, which is funded by an NSF grant and a CACR grant. Am I missing something?

http://icei.org/activities/ntp.html

http://esr.ibiblio.org/?p=6820

> A couple of stories by Charles Babcock and (my coincidentally old friend) Steven J. Vaughan-Nichols have mentioned the existence of an ‘NTPsec’ project being funded by the Core Infrastructure Initiative as an alternative and perhaps eventual replacement for the reference implementation of Network Time Protocol maintained by Harlan Stenn and the Network Time Foundation.

I confirm that NTPsec does exist, and that I am deeply involved in it.

https://www.coreinfrastructure.org/faq

> The Core Infrastructure Initiative is a multi-million dollar project to fund and support critical elements of the global information infrastructure. It is organized by The Linux Foundation and supported by Amazon Web Services, Adobe, Bloomberg, Cisco, Dell, Facebook, Fujitsu, Google, Hitachi, HP, Huawei, IBM, Intel, Microsoft, NetApp, NEC, Qualcomm, RackSpace, salesforce.com, and VMware.

http://www.ntpsec.org

Hilarious reading, of course, "I do have an an advantage because I’m very bright"

There's at least one thing ESR and phk agree on - the autoconf situation.

I'm looking at the list of tests done in current ntpd.

* Checking whether compiler accepts -g * Checking whether compiler supports C89 * Checking if stdlib.h exists * Check if compiler supports -o * Hunts for just about every command line tool, sed, awk, ln, grep * Is size_t a type?

There are dozens of checks that could be replaced with "Checking for OS = Windows".

There are specific code paths for Digital Unix 4.0, a fix specific to AIX 4.3 with the comment "hopefully adding ... fixing this without breaking any other platform", Ultrix, winnt3.5?,

How many NTP developers routinely test new code against all these platforms?

However it's done, more big name programs need to move on from this.

One thing that I find a bit worrying is that there are now two projects that try to create better implementations of the existing ntp protocol, but the protocol itself is inherently insecure. ntp right now is a clear-text protocol with no authentication and no protection at all against man-in-the-middle-attacks. There are two authentication modes from which one is insecure and the other relies on symmetric keys, which is impractical.

There's some work done on ntp security [1], I hope we'll see this implemented soon.

[1] https://tools.ietf.org/html/draft-ietf-ntp-using-nts-for-ntp

I am assuming that when you refer to "the other project trying to create a better implementation" you are referring to ntimed? It seems that you are forgetting about Chrony and OpenNTPd.

A real-time guard can handle most of what it needs: packet-inspection, rate limiting, spotting NTP layer abuse, transport security, authentication, small TCB, real-time, etc. Highly-assured guards have been used for about every other insecure app and protocol so I don't see why not this one.

Throw in tech like Code Pointer Integrity or Softbound on client side for extra measure. Or reimplement it in an inherently safer language.

Reply on news.ycombinator.com