Hacker News Reader

> The article makes a brief reference to DJBDNS, which is written in C

About 14 years ago I worked on a project that was a fork of qmail. I got to know DJB's code quite well, to say the least it is awe-inspiring. The level of understanding and craftsmanship he has in C is honestly something I am certain I will never achieve. At the same time, it is some of (for me) the most dense and obtuse code I've had to read. I'm not a fan of loop unrolling, I think that's a thing for the compiler to optimize personally.

Anyway, in researching before starting this project, I was very aware of DJBDNS and all of the tools that make up that suite of tools. It is solid, like a rock. But like a rock, it is also inflexible. If you want to read an interesting post from DJB, this is excellent one on AXFR/IXFR: http://cr.yp.to/djbdns/axfr-notes.html

In reading this I realized that we have different goals for DNS. I want DNS to be flexible and secure, he clearly wants DNS to be secure and hardened. We have different goals, this is not to say my goals are better or worse, but I do think they differ fundamentally from that of DJB's.

You can't exploit something that you can't change.

You might find this interesting: "However, the closest in functionality and intent is Dan Bernstein’s djbdns, which aims to be a minimalist, highly secure DNS server. The latest release of djbdns, including various support tools, is about 10,000 lines of C as measured by sloccount. We expect that it is possible to build a feature-par version with Nail that is an order of magnitude smaller and intend to do so."

https://people.csail.mit.edu/nickolai/papers/bangert-nail-la...

Nail, btw is parser written in the spirit of "language theoretic security", which is a poweful new way of looking at security problems.

I did mention that flexibility to change is a benefit to a Rust rewrite. Especially to changes made by people with <10 years of experience writing hardened code. :)

> But BIND isn't just failing because "it's written in C", it's failing because it's written in terrible C.

Yup. This has been the case for virtually all of the "core" open-source C projects I've opened up the covers on. In particular, bash's code base is terrifying. The more I look, the more I believe that nearly everything we build on top of needs to be thrown out and rewritten with a safer language (e.g., Rust) and with a focus on incorporating software development best practices we've learned since the past forty years.

"But BIND isn't just failing because "it's written in C", it's failing because it's written in terrible C. "

It's because it's written in C. Writing it in average Java, Go, or Ada would've prevented most of those problems because they're not allowed to do the damage by default. That's because the C language, by design and default, adds risk into common operations that doesn't have to be there. Especially today where PDP-11's no longer dominate hardware constraints. That you had to cite the DNS of an elite cryptographer and programmer to make the point supports ours that writing security-critical software in C is the first mistake. Not spending years of practice, significant time, and using available QA tools in the individual project to correctly write the C is the second.

Contrast the situation to IRONSIDES DNS where a decent, but not elite, coder wrote his DNS in SPARK Ada. It can prove absence of most errors that lead to code injection if you can express the app within its limitations. So, just by avoiding C in favor of SPARK, he gets to say it's immune to single packet DOS plus a number of code injections. Similarly, one would at least have memory-safety (50% less errors than BIND) if they used a Wirth language. Still fast & easy to code, too. BlueJekyll, who I assume is average to above average, used Rust to similarly knock out plenty of memory-safety problems.

As in Orange Book days, the language choice should be considered one of the assurance activities (or lack of assurance). Its ability to securely express the problem, be easily analyzed by verification tools, safely optimized, and securely compiled to ASM are important. The fewer risks in each the better. That C poses anywhere from a high to grand challenge across the board argues against it being a default & for its use being an explicit, security risk.

We will port our security critical code to Rust when you can ship it in Rust for Mac, iOS, Android, Windows, Linux, BSD, FreeRTOS, SYSBIOS, and VxWorks.

C's dominance today is about platform and architectural support, not hardware constraints. It's the JavaScript of systems programming.

A lot of those platforms should be covered; I'm not mega familiar with the RTOses you mention, SH-4 is one arch that I'm pretty sure nobody has done a port of yet, though it seems like LLVM supports it?

Can't you do a reference implementation in a language with great static checks like Rust with an equivalent one in C/C++ side-by-side? From Orange Book A1 to modern seL4, they always did something like that where high-level problem is expressed in analyzable way with equivalent, low-level code. Always found problems testing missed. In your case, SPARK would prevent all kinds of problems with Rust's discipline possibly reducing those temporal errors.

Note: Many safer languages can also compile down to C and integrate it with FFI with type-checked wrappers. Classic strategy for dealing with ecosystem problem. I can't recall if Rust has one. It should.

Note 2: You look to maybe have over-constrained yourself with supported platforms, too. Only four on that list are critical.

What we do right now in our project is write thin C++ but use C++'s features (class types, exceptions, catch all, etc.) to implement bounds checking in parsers, etc., in a relatively safe way and with near-zero cost. If we need to go to plain C later we can "minus minus" this thin C++ code base relatively easily. We could in theory do the same with Rust but it would be more work since it's not as syntactically close.

C++ is available almost everywhere C is available, so it's an acceptable compromise. We also fuzz any "C-like" parser code, of which there is not much.

I am really optimistic about Rust. I think it's the only thing with a chance to displace C in systems coding. (Go is too intrinsically heavy for tiny devices and has other issues.) But it has a long road ahead to work its way into all the corners of computing that we need to target in practice. I wasn't hating on it, just pointing out how hard it is to displace an inferior language with a monstrous install base. Look at JavaScript.

Edit: here's an idea: what about a Rust-to-C transplier? How practical would it be to implement such a thing well enough to allow it to be used for low-level systems programming?

There is a very very early work in progress one, but it's not nearly ready for non-toy programs yet, it doesn't transpile nearly all of C. Gotta start somewhere!

"write thin C++ but use C++'s features "

Well, that's good given I previously told you that your best approach to memory safety... given C++ use... was to convert it to C to run through Softbound+CETS or something similar. Alternatively, a C++ to C compiler then that. I wonder what the state of C++-to-C compilers is these days in terms of what they take in and quality of output.

Rust has zero-overhead calls into C, but no production-ready transpiler yet.

Bernstein used to release all his code under a weird license, by which modified versions could not be redistributed.

This changed in 2007 when he decided to release ALL his software under a public domain license (that includes djbdns obviously)

I'm on a mobile now so I won't lookup the sources for this statement, but I'm sure it's not hard to google...

Found the official statement: https://cr.yp.to/distributors.html

>2007.12.28: I hereby place the djbdns package (in particular, djbdns-1.05.tar.gz, with MD5 checksum 3147c5cd56832aa3b41955c7a51cbeb2) into the public domain. The package is no longer copyrighted.

Great news! Very cool, thanks for the update.

It's obviously been a while since I've run my own Qmail or DNS server. :)

Well, djb's dns has bugs (zone corruption, lack of duplicate outbound surpression leading to trivial poisoning, query pool flushing) and missing essentials (IPv6, and most DNS since 2007). Some of these bugs were paid out in fact. There are patches, but not everything is fixed, and not all the patches play well together. The lack of maintenance and an upstream has caused some distros to consider dropping for security reasons.

I'd agree the C code is easier to read than Bind's heavy use of macros.

Probably the best part of djb's dns was the source port randomization. The logic was key to responding NS eviction attacks (Kaminsky's attack and its variations).

Rewriting things in Rust probably doesn't protect against logic bugs like that. Likely it would introduce new bugs of that kind.

That's a real risk. That plus incompatibilities with real-world software that ignore the specs or get creative with any gaps in it. Those have to be tested. Ideally, if it's a critical protocol, it will be formally specified and verified to ensure the code does exactly what it's supposed to. At the least, formally specified with DbyC annotations and/or tests checking each part of the specs plus restricted, coding style. That should knock out most logic errors.

Modern tooling like Haskell, Dafny and SPARK Ada make this way easier than it use to be. I've recommended doing it in (preferred language) side-by-side with one or more of those in most equivalent way possible so analysis prevents errors.

It would be wonderful if we could get some standardized language for defining test cases, and some tooling for converting to working (or mostly working) code for your language and implementation (at the API level, at least). If publicly developed per protocol/library, I think it would yield a substantial benefit for robustness of common protocols we have.

Want to develop a DNS library or server? Read the RFCs and download the public test suite (which may get regular updates as people fill out corner cases, put in tests for common bugs/exploits, etc). Occasionally run public test suite plus any of your own tests you have added to confirm things function as you expect. There's so much duplication of effort in this area currently, and i'm sure the majority of test suites for related applications have at least a few tests that would be useful to others.

I agree it would be useful to have. There's all kinds of stuff like that in academia. So many different forms and tools that the problem would likely be the time available to assess them. On top of nobody using or replicating them as usual. :)

Note: sklogic did mention doing something like that by making sure all the programs' had a scriptable API then using basic, shell input and output to test them. There were acceptance, testing tools I used a long time ago that did something similar using XML. Not sure the efficiency of this method but it should work.

There are not many macros in BIND: mostly assertions, debug log abbreviation, and BSD style linked lists.

> which makes it easier to extend without adding security holes on a weekly basis (coughBINDcoughOPENSSLcough)...

Worth remembering that until the real crypto libraries arrive in pure rust, openssl is still being used internally.

Yes, though also ring, which is derived from OpenSSL through boringssl.

> But BIND isn't just failing because "it's written in C", it's failing because it's written in terrible C. That said, "terrible C" is probably most every C routine written by someone with less than 10 years of solid low level experience, so "writing good C code" is not very scalable.

Please do remember that many of these codebases are ancient.

If you rewrote many of these in C today, they would be much better--especially if you also added a test suite around them.

I'm happy to see this in Rust as I'd like Rust to get some real, hostile-environment usage to see what breaks.

I thought DJB generally donates his code to the public domain. Hard to be "friendlier" than that.

Public domain code is actually illegal in some places, if I recall correctly

Not quite. As legal explained to me -- it's too legally ambiguous for some entities to feel comfortable using because your only fall back on public domain is copyright, which can be overly restrictive, and the usage rights are undefined and enforcement or rights can be applied or modified at will by the copyright holder in the absence of a clear license. And that copyright, even in the public domain, often exists inherently.

That's not an issue for me at my $corporation because they are sane, but apparently that's the argument.

No illegal per se, would be closer to say it's undefined or that it has no value. IIRC Germany doesn't allow you to disclaim your copyright. A CC0 dedication is probably the best option for a portable public domain option.

Here in Sweden, it is not illegal - it is more that there is no legal concept of a public domain which makes any license that try to transfer something to the public domain be nonsense and invalid.

> DJBDNS hasn't been updated in some time, so a more modern project isn't a bad idea.

How so? If it ain't broke...

The generalizations about the quality of C code is interesting...It is easy to criticize code that has been developed over years, multiple hands, etc.

C code doesn't have to be terrible. And "terrible" often is in the eye of the beholder.

I've heard people decry OpenSSL and it's various warts (it has many), but as a code base, it's worked well for years across an incredible range of platforms.

Some claim the architecture and quality is crap (often, it's more about style), some just know that's how it was developed and stick to the style and extend / fix / etc. A lot of the OpenSSL issues stem from the wide range of use and the lack of giveback by companies working with it (including donations, etc. This goes back nearly 15+ years).

A new and hot language does not make a new implementation any more secure (maybe it has different issues) than one that has been around awhile, had multiple eyes, etc.

> I've heard people decry OpenSSL and it's various warts (it has many), but as a code base, it's worked well for years across an incredible range of platforms.

That... really depends on what you consider "worked well". Being a cryptography library, I think I could make a strong argument that working well includes a very good track record with regard to exploits. I don't believe OpenSSL exhibits that[1]. Note the number of items that include overflows and memory corruption in that reference. Also note that the first page (50 items, but some just DoS) only takes us back to mid 2014.

> A new and hot language does not make a new implementation any more secure (maybe it has different issues) than one that has been around awhile, had multiple eyes, etc.

That's true. Unfortunately, OpenSSL has proven itself to be a poor choice for a cryptography library for those that value security over performance. We used it because it's what we had that was free and supported what we needed, but we need more competition (which we are finally getting) so people can make decisions based on their priorities, not just the small pool of what's available.

New implementations (by knowledgeable security professionals) are what we need. Whether those are done in C/C++ using more modern techniques, or some other language that can eliminate certain classes of error in some other way is somewhat irrelevant, as long as it works as a library. I suspect that the vast majority of people would take a 50% performance hit if it yielded a library that had only half as many major flaws as we've seen. That we might be able to approach that with much smaller performance loss is exciting.

1: https://www.cvedetails.com/vulnerability-list/vendor_id-217/...

> I suspect that the vast majority of people would take a 50% performance hit if it yielded a library that had only half as many major flaws as we've seen

How would one know whether there are any major flaws in a code base?

Openssl very much falls into the "so complicated / complected that there are no obvious flaws"-bin, while something like NaCl is closer to "So simple there are obviously no flaws" one.

Note that there turned out to be a distressing number of obvious bugs in Openssl, and that it's doubtful that all serious bugs in a full, real-world crypto stack (especially perhaps if you demand support for x509) could ever be obvious.

You don't, for sure. But when half the security bugs seem to be related to memory corruption and buffer overflows, there are options. We've had languages that put more emphasis on being correct and safe than on performance, but people haven't been using them for much. C/C++ have momentum and people that know how to use them, so they are used. I'm not sure that makes them a good choice for security related work, where bugs sometimes counter the entire purpose of the library.

> C code doesn't have to be terrible.

True, but I hardly have seen any codebase at enterprise level that isn't so.

C doesn't scale when you have projects where modules are written all the time by different people that got hired to work just on piece A and then leave.

They hardly can keep on their head all the nuances of memory management and UB of the company's codebase they have been asked to develop the feature A for in a month.

Languages with more strict type safety help keep people on the right path, even though they still stumble every now and then.

> C doesn't scale when you have projects where modules are written all the time by different people that got hired to work just on piece A and then leave.

That is true for any language, C, Java, Rust, Go. Continuity from team members is important.

No, because some languages make it easier to deal with developers as if they were cogs (even if I hate how things are nowadays).

For example, the use of automatic memory management or bounds checking, not only allows to remove an whole class of errors they remove the need to have a deep understanding of memory allocation patterns in the code.

Similarly the way one can make use of strong type definitions like in Ada, ML languages to only allow the code to compile if certain rules are followed.

We all hate to be dealt as cogs, but that is how the enterprise world works, the majority of the jobs are consulting gigs for specific features in short to mid term projects.

Perhaps, but being a cog in the software team at entreprise level does not preclude problems such as understanding the design decisions made or system limitations that are placed on the software design etc. Sometimes the inscalabity of a design is language independent. The problems are probably emphasised when consultants are engaged to do the job.

I agree, just making the point that languages like Ada or Haskell also have features that make certain types of errors impossible in such scenarios.

Of course there are other types of errors that they still cannot avoid.

Minor fix, "has suffered zero security exploits" is really "had zero security vulnerability". Exploits are the tools/code using vulnerabilities. Software has security vulnerabilities or security bugs.

The "suffered" language makes it work. E.g. "Has suffered zero fools." Fools are people you don't want to deal with. Your problem if you suffer them might be you are too open to dealing with people wasting your time. You could say you had zero problems being too open to people wasting your time, or that you suffered zero fools, and they are roughly equivalent.