The djb way (2007) | Hacker News Reader

The djb way (2007)

thedjbway.b0llix.net

62 pointsvezzy-fnord41 comments

One thing to learn from djb is how to write virtually bug-free code.

Much more people would be able to do that if they:

a. Stopped using buggy dependencies b. Focused on the task being solved instead of on the technology

Except, not many people are as smart as djb. But, djb being smart doesn't automatically make his tools good ideas to use (I'm looking at you, Mr. "I made my own register allocator because I don't trust compilers").

djb lives in a quaint fantasy toy land where he gets to write whatever he wants and doesn't have to interface with real world systems at scale. That's great when you're the author. Your software can be clean if you only write for yourself and not for 50 million other programmers trying to build software on top of your tools as well. But, it's painful when you're a user and real world problems get ignored because the author just never experiences your same problems (so they never care to fix them).

Much like how a startup isn't 90% code and 10% "company," public software in 2015 shouldn't be 100% writing code all day then dropping .tar.gz files on a server once every two years while ignoring wider principles of project governance, collaboration, and feedback/growth processes.

In 2015 open source software development is 90% about doing peripheral work, sit in committees of various foundations, force people to use a different infrastructure every two years, and generally, avoiding real work as much as possible.

In practice, you'd be using one of the many successors to djbware today, not the direct originals. You'd install runit, s6 or daemontools-encore rather than plain daemontools. They all reuse the same libdjb interfaces and follow the same code style, otherwise functionally they're supersets.

I use void linux (http://voidlinux.eu) on my laptop and vps, which uses runit as its init system, and it's beautiful. It's such an elegant system. Writing the run scripts is so easy, and there's so little you need to understand to do it.

Rather than learning how upstart or systemd unit files work and learning their configuration languages, you only need to know how to start a process that runs in the foreground and logs to stdout.

Here's a template for doing the same in systemd:

  [Service]
  ExecStart=*path to process*

Learning the language is only necessary because some services are more complex. For example, if you need your process to only start after another, you must now learn the syntax of runit's sv program.

Here are some concrete run scripts and service units, side by side: http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/ru...

I was curious if such a thing existed, thanks.

Bernsteins programs are definitively a bit... different. For some reason the way you use/administer them always seems more sensible to me. I believe that many of the complaint people have is a reflection of configuration being different from what they'd expect.

Publicfile and ucspi-tcp might not work "at scale", but qmail, ezmlm, djbdns and daemon-tools certainly work even for very large deployments. I would agree that qmail and ezmlm benefits from the patch-set floating around.

So what are they using for their webserver? [edit] http://thedjbway.b0llix.net/services/mathopd.html

It seems interesting in the same strange way http://suckless.org is interesting. A certain pleasure can be found in running a website as a 300 line C program, but it's not very useful besides that.

It seems pretty useful that despite being a decade-old installation and being linked to from HN, it is responding near-instantaneously.

Likewise, the suckless tools are useful in that they just work. That's pretty useful too.

It sometimes feels to me that we're missing some essential component of the software creation toolchain: a configuration-time or link-load-time equivalent to a compiler's "dead-code elimination" or a linker's "whole-program optimization" or a runtime's profiling JIT.

In other words, presuming a given program was shipped to you as a source package, your package management system could contain a pass from a generic program that takes the software "seed"—the generic version that does a lot of things—reads a configuration file specifying your particular use-case—and then hardcodes policy-level choices like protocol flags, strips out every code path that you won't be invoking, removes conditionals altogether where only a single path is left, inlines functions that have then been made trivial, etc.

It would be sort of like a version of autoconf that works at an AST level; but also built on the expectation that you'll be able to run it again to generate a new binary if your business needs change, rather than just serving to enable you to cook up your program with whatever nasty ingredients are left in your OS's fridge.

The result of such a source-transform pass should be something like one of the djb programs: a terse piece of code to achieve your particular goal. Ideally, you would then take that program and read it before using, because, well, if your particular goals are reasonable, then the code should be small enough to fit in your head!

On a tangent, the "unikernel" philosophy, as I understand it, is simply that if you could throw the entire kernel + OS + your application into such a goal-oriented streamlining woodchipper, a unikernel application is what would result.

That is some hagiographic religious bafflegab.

And with good reason. I don't know where things are with his software and methods today but, last I heard, djb's software and methods are world class ideas that everyone should be using.

Please don't use his ideas about human-unreadable log files...

Look, djb is a damn good programmer and writes some amazing, reliable software. It's just not all that user or admin friendly. I'd go mad in a world where everyone used his 'world class' ideas on software...

I presume you're talking about the use of TAI64N timestamps. Making them human-readable is a pipe to tai64nlocal away.

djb's software is, in fact, quite user and admin-friendly. However, most people simply aren't used to the modular toolkit approach way of doing things beyond basic text processing. The djb philosophy is really the Unix philosophy, and so it comes off as quite shocking to see software written like that when you're used to monolithic blobs.

But there is so much insight to be gleaned that, unfortunately, most people ignore precisely because to them user friendliness means building inflexible kitchen sinks. I especially do not consider the latter to be admin-friendly in the slightest.

It's IMO the wrong way around. Logs should be human-readable without needing a separate program. Computer code can convert a human-friendly timestamp into a more CPU-friendly one with minimal effort. We have these processors that can do it, so why not let them? Why does djb choose to make life difficult for humans?

http://harmful.cat-v.org/cat-v/unix_prog_design.pdf

Because it lets the implementation be independently reusable as a filter. Furthermore, there are some caveats about converting TAI64N to localtime, so having the admin be aware is desirable.

Having to pipe a program isn't "making life difficult for humans," I have no idea how you came to that conclusion.

An odd choice of document to back up your argument. I can't think of any unix programs that use anything other than human readable dates and times in their outputs, for instance.

Further, nothing is stopping any implementation be independently reusable, whatever the choice of format. That's an irrelevance.

Essentially, it's not a technical decision. Trying to back up your preferences with tech reasons is besides the point. I just think that anything that makes it more difficult for a person to read the log files is a bad choice. If you can choose who has to do the extra work, human or computer, you should always let the computer do it.

> I can't think of any unix programs that use anything other than human readable dates and times in their outputs, for instance.

Maybe, but look at the date display code in the GNU core-utils for instance, it depends on the locale and everything, not quite minimalistic.

I understand your point but it is inconsistent with the rest of djb's way. It does not mean that it is invalid in general, but that this "human-unreadable" log files in djb's code are actually a good idea in the mindset/framework of djb's code.

There are many format and human language you could write the date into. And many timezone.

By putting timestamps in log files, and requiring humans to type a few more keys to transform it when they need to, every one can choose its date format, timezone, and language. So, to me, djb's way is the human-friendly way.

I agree. Further, all kinds of tools use logs in different ways and often need decent CPU power anyway. Making logs efficient & machine-readable by default aids that. It also lets me use lower-cost components for services producing and storing them. This goes same for lightweight, high-performance services like we've seen show up in HTTP. So, instead of one bulky server for several grand I might have several, embedded boards in a H.A. configuration with plenty of memory and storage doing same workload.

So, efficiency still matters today if you're squeezing the most out of your systems or like physical isolation like I do. Easy, portable processing on arbitrary machines, too. Human readable logs can have an impact on that. A nice compromise, though, is stuff like JSON, s-expressions, or a TCL style with abbreviated keywords.

Guess you haven't looked into how so much of Linux is migrating the opposite way lately? I get the irrational fear of losing plain text, I shared it too with HTTP/2 (as one example), but the fact of the matter is once you use it for a short duration, you realize it's just as simple and useable.

So, it's user-friendly for some hypothetical set of users that is not the set of users that currently exists.

The term itself is devoid of meaning altogether, so it is ultimately a diversion anyway.

So in the space of four hours you've gone from "djb's software is, in fact, quite user and admin-friendly" to "[t]he term itself is devoid of meaning altogether?" Talk about moving the goalposts.

No, it's consistent. It's a meaningless dog whistle like "socialism". Depending on your ideological perspective, anyone from a hardened Maoist to a Chicago school economist could qualify, or anything to do with state control whatsoever.

Therefore I'm really flipping your buzzword and using it against you. There are no HIGs for system software, and more often than not "This is user friendly." is a codeword for "I'm biased into thinking this way." It's not a criticism, it's a thought-terminating cliche.

If you have technical criticisms of djbware, that'd be fine. You evidently do not.

(That said, I do think it's generally accepted that system software disobeying you by making policy decisions you have not authorized is not admin-friendly. djbware excels at being mechanism only.)

Where is it MY buzzword? You used it before I did, my first comment in this whole thread was a response to you.

joosters'. But you reiterated on it.

So you cleverly decided to turn jooster's argument around on me because you couldn't tell the two of us apart?

So, it's user-friendly for some hypothetical set of users that is not the set of users that currently exists.

That is a reiteration.

Either way, neither of us are saying anything of substance.

That's quite some hyperbole, but, well, you don't have to use it. Like the old saw, "Unix is user-friendly, it's just selective about who its friends are."

Well now you get systemd, with the fun of binary logs but without the benefit of being written by djb.

djb's software is principled and very well engineered, but a document that appeals to MIND-BURSTING POWER OF PURE REASON can't really then turn around and present a faith-based argument for doing something.

His software creates new top-level directories off root and this simply hasn't found wide acceptance among sysadmins. I've never seen eg daemontools used in the wild for this reason.

Netflix uses daemontools in their BaseAMI[1] for starting up core services such as Apache and Tomcat. I think this choice was originally made to ensure consistency in process startup methods across CentOS and Ubuntu. Support for automatically creating daemontools scripts is baked into nebula-ospackage-plugin[2].

[1] http://techblog.netflix.com/2013/03/ami-creation-with-aminat...

[2] https://github.com/nebula-plugins/nebula-ospackage-plugin

Yelp uses s6. At least, I'm deducing that from employees being regulars on the supervision mailing list.

It occurs to me that the configuration-time and maintenance-time "weirdness" of djb's software is a lot less of a big deal if you stick it in a Docker container. I've seen daemontools being used as a container-internal PID1 a number of times.

In college we had a class about djb oriented design.

Took me a while to notice that it's not written by djb himself.

It should not have. Bernstein writing about his own designs has a markedly different appearance.

* http://cr.yp.to/qmail/qmailsec-20071101.pdf

* https://news.ycombinator.com/item?id=9250118

Even to the writing of Aaron Swartz.

* https://news.ycombinator.com/item?id=890034

* https://news.ycombinator.com/item?id=9155741

Reply on news.ycombinator.com