Why is the OpenBSD documentation so good? | Hacker News Reader

Why is the OpenBSD documentation so good?

dataswamp.org

213 pointszdw91 comments

My perspective is skewed since I only get the questions that the documentation doesn’t answer, but I support someone who uses OpenBSD and the documentation is far from perfect. Some complaints that spring immediately to mind:

The man pages sometimes document things only by implication, or with confusing wording, such that if you already know the answer it makes sense, but if you don’t it’s near-impossible to find out. (To be fair, this affliction is definitely not exclusive to OpenBSD’s manpages.)

From what I can figure out, their 'ksh' is a vendored copy that they forked and made a bunch of changes to without ever updating the version string, so despite claiming to be “PD KSH v5.2.14 99/07/13.2” it’s got 20 years of patches added on top and won’t match any ksh you’ll find anywhere else. This is not explained anywhere that I could find.

There are some questions in the installer about 802.11 setup which are terse to the point of incomprehensibility and AFAICT entirely undocumented. (I had to go rummaging around in the source code and read the installer script to figure out what it was trying to get me to do.)

A particular peeve is that the sh and ksh manpages are completely different, despite documenting the same program. (They’re the same binary, with the non-POSIX bits left out of the sh manpage.) It’s particularly frustrating that there are some things documented well in one but poorly in the other, so once you’ve found the same documentation on both sides you still have to figure out what’s actual differences and what’s just bad phrasing.

OpenBSD’s maintainers also seem very insular and their mailing lists have a reputation for being very abrasive, so it’s also very intimidating to propose improvements; as a result these papercuts stick around (not being very obvious to the developers, who already know how these things work and so miss them) and continue to mar the polish.

> such that if you already know the answer it makes sense, but if you don’t it’s near-impossible to find out.

That was my experience, not being familiar with unix / Linux - but the https://www.openbsd.org/faq/ does go along way to bridge the gap between the man pages and “How can this OS be configured to do what I want?”.

That is what I find with much Unix and emacs documentation. It is for people who already know the tools, there aren't the hooks whilst you are learning.

The documentation I found much better was MS in the 1990s where you git an overview as well as the details.

Apple documentation used to be quite good but all their overview docs are now in archive and not updated and you just get the API.

On the mailing list don't take anything personally and don't try to "feel the room" and you should be fine.

Keep it at a pure technical level. It helped me when I had answers that I would have considered rude but where technically correct.

> The man pages sometimes document things only by implication, or with confusing wording, such that if you already know the answer it makes sense, but if you don’t it’s near-impossible to find out. (To be fair, this affliction is definitely not exclusive to OpenBSD’s manpages.)

Man pages were not meant to be the only documentation except in the case of simple programs. For everything else man pages were meant to be a concise reference for people who already know how to use the program but needed to jog their memory about some aspect of it.

If you needed to learn something complicated, such as sed or awk, you weren't supposed to try to learn from the man pages. You were supposed to reach for your copy of the Unix Programmer's Manual.

...which is a mistake. There are four kinds of documentation (https://documentation.divio.com/), and splitting up those four kinds up among different places is a very bad decision.

Man pages contain at least reference information, but they're very inconsistent about including how-to guides and and explanations, and almost never contain tutorials.

Big mistake.

I believe that info was meant to fill that niche, but it appears that most non Emacs users have never heard of them.

Don't let the bad rap hinders you. That has not been my (small) experience with the mailing lists.

You propose a change:

- if it's interesting, you will get advice on how to make it good

- if it's good it will get committed

- if it's not relevant, you will be told

Here are some examples: https://marc.info/?l=openbsd-tech&m=155934125101222&w=2 https://marc.info/?l=openbsd-tech&m=156256473722583&w=2

> their mailing lists have a reputation for being very abrasive, so it’s also very intimidating to propose improvement

What reaction did you get when you proposed a documentation patch?

I’ve suggested this, but the person in question has done enough reading of the mailing lists that I trust their judgement when they say they don’t want to get involved. (Sorry for being so vague here; not being an OpenBSD user myself I don’t really know the details.)

FreeBSD handbook is also very, very good. I think that the BSD's are each more or less self-contained has something to do with it. You can read everything about system services, for example, in the handbook without having to chase down systemd blog posts and compensate for various differences between versions and distros.

They are indeed very good but the pace of change being slower than Linux also makes it easier to keep things updated.

It's also that the process of change requires documentation. You won't get changes committed without it.

That seems like the root cause here. Why on earth isn't that the standard everywhere?

Because writing documentation isn't easy, and a lot of people who like writing code don't much like writing docs. So for an open source project "the documentation matters" has to be a conscious choice (a cultural value, if you like), which is actively enforced by (a) the regular devs pushing back on contributions which don't have docs and (b) those devs demonstrating a commitment to the value by upholding it themselves, ie writing docs for their own contributions.

Obviously "better docs" would be better for the users of software, but most people contributing to most open source projects (or indeed to most non-open source software!) are not doing so primarily for the altruistic goal of making users' lives easier. So the default position ends up being "do the stuff you find fun" or "do the stuff that your employer cares most about" or "do the stuff that fixes problems you personally are running into", which will often not be "great docs".

(I speak as somebody who's involved with an open source project whose documentation is overall not all that great and which doesn't have a "new changes must have docs" rule, so I don't mean to throw stones here -- just trying to argue that you should probably expect that poor docs are the norm and good ones the rarity.)

What if the slower pace is a conscious choice that makes the resulting documentation better and codebase more solid?

It's not (a conscious choice). There just are fewer people working on BSD. Linux has to handle an avalanche of developers wanting to merge their code.

Not being a disjointed mess of unrelated software helps too.

Any specific examples?

Every Linux distribution.

It’s decent but nowhere half as good as OpenBSDs documentation. Not complaining but it’s just not kept up to date in some places. Use of Jails for modern containers (eg using iocage) is a good example.

iocage is not part of the basesystem.

It's a bad example.

From a FreeBSD comment I made the other day:

> If you want to try FreeBSD just use the Handbook, it documents FreeBSD very well. That said betrays childish aspects e.g. they refuse to list that FreeBSD can run on Linux/KVM (the mere fact it can, I'm not talking in depth instructions).

(from id=32369189#32372408)

It lists KVM as an option in the very first part of Chapter 2 under the synopsis.

https://docs.freebsd.org/en/books/handbook/book/#bsdinstall

This is what I was talking about:

https://docs.freebsd.org/en/books/handbook/virtualization/

goes way back... (and into the contained bug report)

https://forums.freebsd.org/threads/please-give-some-ideas-ab...

It really, really is.

The comprehensive and well-written man-pages are one of the reasons why OpenBSD feels so "complete". In German, you'd say "aus einem Guss", which almost literally translated means "from a single mold".

Linux often feels like a bunch of different things thrown together (which it always was), and the quality, or even just existence, of the documentation differs accordingly.

This echoes of cathedral vs bazaar…

My thoughts exactly! I'm sick of Linux's disjointedness and impossible to follow OS. I'll see you guys on Windows 11. Peace.

I wouldn't go that far... I've used Linux since the mid-90s, and Windows for a few years longer than that, and Linux is still orders of magnitude easier to follow and especially to dissect than Windows. Windows is pretty much entirely opaque, you're at the mercy of whatever happens behind the curtain. UNIX-y OSes like Linux and OpenBSD blow that curtain wide open, but OpenBSD is just much more consistent because it's all built "in-house". (And then some super popular stuff like OpenSSH gets exported from it.)

At the risk of being hung, drawn, and quartered on hackernews, I actually agree. This seems like such a fringe idea among the "hacker" culture but I find Windows to be significantly more tolerable than the not-so-organized chaos that is modern Linux. And BSD tends not to focus on the out-of-the-box experience, but more on the "customize it exactly the way you want it", archlinux-y vibe. Which is cool but I'd rather have something that just works. Windows checks all the boxes for me, was free through my university, and if I get tired of Powershell I can use WSL.

To compare and contrast, when I was trying to get wifi up and working on my UltraSparc laptop, I read the OpenBSD man page for ipconfig. While very long, it got me up and running in short order. No Google required.

On the Linux side, there was a Linux utility mentioned on HN and it was broadly praised. So one "pamac install" later, I was reading its man page. I had to resort to Google because I couldn't even tell what the program was supposed to do from its own manual.

Well this is my experience with almost all software.. except with Ruby on Rails.

In the good 'ole days, I'd work completely offline, using only the documentation provided by Microsoft in Visual Studio. It was complete, correct, and had samples. No distractions, not googling, and no HN out of the blue!

Hah, and now the Windows help just opens a bing search results page in your browser. It is like they've fully capitulated. I find opening the browser to a search engine is very condescending; if I wanted to do that I very well could.

Always look for adequate docs before adopting any library into a project. It is a lesson you need taught only once.

Windows help was always bad. You could not find anything in it. And now they made the logical step: they made it worse. I think it follows the UI guidelines.

But the gpp wasn't talking about Windows help. He said documentation and Visual Studio; I presume he's talking about the MSDN CDROM that came with a VS installation. You could optionally access the docs from the CD or install it to your hard drive for speedier access.

And it was quite good. You could learn just about everything you needed to work from it, offline and from one source.

There are many things I did not like about Microsoft back then but their developer ecosystem was not one of them. Tools were expensive and closed but quite complete and well put together.

Until I learned about open source and free software I considered MSDN to be the pinnacle of how software development should be.

These days it just feels like MS can't decide between free or prosperity and so can't seem to get either one right.

Samples, sure, but they were written in weird Microsoft style and made very little sense to a young me

Also, another H/T for Rails docs, they are very good... but far from comprehensive

Because they understand the value it holds and put in the work to maintain it.

Documentation is expensive. Good documentation is more expensive, requires different skillsets than most engineers have, and requires a bit of an organizational commitment to make work.

That last part is more difficult in Linux-land than BSD-land, which is only a partial excuse. The main reason it almost universally sucks is that people put up with it, so managers are more than happy to cut those expensive tech writers.

I think it has more to do with the lack of metrics to be defined — and thus tracked. It’s easy enough to track features, and even code quality (by # of bug reports); you can track documentation by “coverage” — each feature has some kind of page — but it’s difficult to define and track for documentation being comprehensive and well-written.

And if it’s not tracked… it’s invisible

So your argument boils down to the incompetence of the Linux projects then?

Incompetence of software projects across the board, except in a few rare cases, sure; more importantly, it’s got nothing to do with cost optimization (or malicious MBAs)

BSD folks just want to make the world a better place. And I love them for it.

Most official GNU project software usually has pretty solid GNU info documentation. But, of course, GNU/Linux has a lot of major not-GNU pieces like the Linux kernel and systemd.

I can't place it, but I seem to remember Richard Stallman stating in some early GNU project goals that he'd seen much software usefulness hindered by poor documentation and made it a notable priority for the GNU project to have good documentation from the beginning.

GNU manuals online: https://www.gnu.org/manual/

The bifurcation of GNU documentation between info pages and man pages is endlessly frustrating. WHY do we need a separate info system? Just put the relevant information into the man page.

Man pages were a Un*x thing. GNU wasn't supposed to have bifurcated documentation: its official doc format was (is) texinfo, from which info pages and printed manuals are generated. People kept writing man pages anway, of course. But the proper question is "WHY do we need a separate man page system? Just put the relevant information into the texinfo manual".

Particularly: man pages were fine for PDP-11 Unix utilities, when they were formatted with nroff/troff and the printed versious usually literally fit onto one page (a few of them ran over into two pages). So they don't have any navigation features, TOC's, indices, or whatever. That fails badly when the man page is large: I see rsync's is 4341 lines (72 pages) and bash's is over 100 pages. There are many other large ones as well.

The only way you can find stuff in a large man page is search the text, which is a pain because there is no metadata to distinguish the search hits. Texinfo on the other hand produces navigable hypertext doc and an organized printed manual with an index, TOC, cross-references with page numbers, and so on. So it is a much superior format.

The problem with Texinfo is that it predated HTML and is now thoroughly obsolescent. There is value in being able to generate a printed book, and to have other formats. But the generated Info format is completely eclipsed by HTML. It existed in a world where we weren't allowed to have proper figures or tables, and linking was very limited. A world like a traditional Emacs buffer. The "info" viewer was despised, and with good reason.

None of this is not to say that GNU didn't do a good job with documentation. The documentation is often excellent. But the documentation format and tooling needed replacing two decades ago. Texinfo is an anachronism. It's painful to author (I have written a Texinfo manual, later replaced by DocBook) and limited in its output capabilities. But today we have Sphinx and other tools which are greatly superior in their ease of use and capabilities.

While in some respects Texinfo was more structured than troff dot codes, it was at the same time vastly less flexible with its formatting, figures, tables and equations. troff let you use tbl, pic and eqn etc. I personally find it easier to search a single massive manpage than hunt through all of the Info nodes. The nagivigability of Info documents is atrocious.

But that's all ancient history. Both troff and Texinfo are several decades out of date at this point. Arguing which is better is pointless when both have been eclipsed by far superior replacements. Which can generate manpages, HTML and PDFs with ease.

Mandoc it's far easier than plain Troff/Groff and it can export to PDF and/or HTML5 perfectly.

It all sounds great in terms of a format. But if it is so superior, how come even on the GNU “home turf” we are still mainly using man pages in 2022? Is it not a classic example of “worse is better”? Where man may have major obvious flaws, but it is simply just good and standardised enough so that texinfo simply can not displace it despite its advantages.

I think it's definitely a case of "worse is better", not because it's "good enough" but because man pages are just "plain text" without the ToC or hyperlink features of texinfo. There is no interface, it's just blasted into a pager and you use the familiar interface of less or more or whatever to navigate.

It's the difference between man/less showing '(press h for help or q to quit)' and Info showing 'Type H for help, h for tutorial.' I accept Info's technical superiority, but I've never intentionally used it because man/less works well enough without having to learn a new interface.

(the other elephant in the room is HTML/web browsers, which I think eclipse Info in reach and familiarity so much that no-one would want to learn a secondary rich-text hyperlinked document viewer)

An interesting tangent to this. The openbsd manual renderer (mandoc) creates less tags which can help on some of the bigger pages (ksh, pf.conf, ifconfig)

An example: to get to the inet option in ifconfig, the normal '/inet' search will have a lot of false positives. but the tag ':tinet' search goes to the item in question.

Nothing against gnu info pages but I always preferred man pages as it feels like less cognitive effort to use. The less tags are indicative of openbsd culture where they try to improve existing systems. linux(due to it's development model) has a tricky problem where it is very hard to improve anything. you can only create something new and abandon the old.

TIL about "less tags"

This is a big help in reading openbsd docs, where they tend to put a lot in one page. For example, linux will split up the openssl man pages, one for each subcommand. OpenBSD has one page. With less as my PAGER, hitting :tx509 makes that a lot easier. Also discovered you can do this:

    man -O tag=x509 openssl

Thanks!

One issue is that the main info browser is in Emacs, so if you're not an Emacs user, Info pages aren't that accessible. There is a standalone info browser but it's not as nice as the Emacs one. And there are html converters but those are even worse. Back in the day, I doubt that it occurred to anyone that a lot of GNU users might someday not use Emacs.

I do constantly use the Info documentation for Org-mode and a few other things, but it is observable (and annoying) that some large GNU programs like bash are documented only as man pages afaict.

The standalone info browser really isn't that bad and neither are the generated html page(s) (especially now that they applied some light CSS) and also the generated pdf look fairly good.

GNU bash also has fairly good info documentation.

I recently decided to finally spend roughly 5 minutes to learn how to navigate around info manuals.

Afterwards I was like, wow, where has this been all my life???

So my answer to your question would be inertia.

Because the info pages (although claimed otherwise) had the same info like man pages, with very few exceptions. And navigating info pages was a pain.

As much as I love Debian, I wish it had better documentation. Having said that I haven’t tried to contribute to it at all, so I guess I can’t complain.

If you install dwww, all of the documentation installed on the system is made accessible from a local web server. Not sure how secure that is these days but I thought it was really cool 20 years ago and it very useful to me.

https://www.debian.org/doc/

/usr/share/doc

man pages. No one uses info pages, whoever says he does is lying.

Two reasons:

1. In general, new programs, and updates to extending programs, tend not to get committed without the requisite manual pages or related manual page updates.

2. "Bugs" in OpenBSD's manual pages are treated no differently than bugs in other areas of the source tree. Find a problem in a manual page? Submit a diff file for it and it'll get reviewed and committed.

Are there any major free/OS projects for which (2) is different? Some projects accept next to no third-party diffs, other projects will accept documentation diffs as eagerly as code diffs. I've never heard of a project that is happy to take code but not docs.

The OBSD docs are good... if you know what you're looking for and are familiar with UNIX already. For a beginner, they are cumbersome, hidden, and annoying to parse.

Good docs look like Tailscale[0] and that similar style where it has the Getting Started at the beginning, along with a nice menu tree on the side that gives an overview and provides easy navigation. You shouldn't need to know what you are looking for when you are learning a platform.

[0] https://tailscale.com/kb/

This article only convinces me that the OpenBSD documentation is bad, for three reasons.

First, there are seven listed sources for documentation. Seven places I might have to look through to get the answers I need. I cannot imagine a reason to have more than two - "static" documentation (the four kinds of docs in the Divio documentation system) and release notes.

Second, there's no mention of awareness of the distinction between the four kinds of documentation (https://documentation.divio.com/), and lots of discussion on the implementation details of man pages but nothing on the actual content. Man pages are pretty well-known for being decent references, but often lacking how-to guides and explanation, and having terrible/nonexistent tutorial information. This makes me think that the author is confusing "documentation" with "reference manuals".

Third, the lack of a community wiki is explicitly counted as a feature. It's not. Users need a place where they can collaborate with other users, and quickly write down their observations and hacks and problems while they're trying to get their job done - they don't have time to start a lengthy conversation with the mailing list developers or go through an involved process for patching the official documentation. Source: me, a user who has to work in a corporate environment and whose job is not working on OpenBSD, as well as many of my co-workers who often don't even take a few minutes to edit the wiki when they find a problem or fix.

In part because they do it for the love of the OS. Now if only they would support these "mongrel" video cards...

As someone who has spent quite a bit of time documenting the internal workings (i.e. design) of various software and systems and much less so user interfaces (ain't nobody got time for that), it is interesting to note that the opposite is what is available in open source projects.

I mean, it makes sense since the internals are probably moving more than the interfaces and since everybody reads code anyway they can just do that. On the other hand, design documentation allows one to quickly figure out how a particular feature works, which is more tedious to do from just reading code. Barrier for use is lower, but barrier for contributing is higher, or something...

Simple answer, because it's a single OS with a single focus and centralized development. Compare the Arch WIKI to the OpenBSD docs and marvel at the difference in scale.

There are downsides to it too, Linux has a lot faster development and broader compatibility but this also results in more complex amd harder to maintain docs.

BSDs tend to have pretty good docs in general. Take a look at NetBSD's internals for instance: https://www.netbsd.org/docs/internals/en/index.html

Does Advanced Programming for the Unix Environment still apply for NetBSD?

It should. NetBSD is relatively conservative.

How often does it have to be updated/changed?

Who's updating it? Volunteers, paid contributors, developers?

possibly for the same reason the Arch wiki is so good.

Arch wiki is precious. But the reason for quality of OpenBSD documentation is unlikely to be the same as Arch wiki's. The article even mentions the problem with wiki's. Arch wiki clearly shows signs that it is collected together by people looking for solutions. Articles often contains discussions on contentious or confusing issues. OpenBSD documentation on the other hand, is prepared along with the application source, often by the same developers. That is a much better solution IMO to ensure documentation quality.

That said, I'm happy that Arch wiki exists. It has become the defacto place where people elaborate their solutions for Linux.

Maybe if you're only looking for a 2 liner to uncritically copy and paste...

I’m happy the title isn’t “The unreasonable effectiveness of the OpenBSD’a documentation”.

i use arch btw

Fedora for life, but I do love Arch. The wiki is unbeatable.

It is not, it's is terrible.

Not even going to click, The BSDs have better documentation because their business model does not depend on selling texinfo manuals.

The texinfo manuals are great, but many Linux distros (Debian and it's derivatives) don't bundle them due to the GFDL license.

Which is a shame. Some of the core pieces of GNU are really well documented, but people who start out in Debian dont realize this and are left reading mediocre man pages and outdated wikis on Debian.org.

The best thing would be to do it like OpenBSD and not have outdated man-pages...

Totally agree. I guess some distros think having no documentation is worse than having bad documentation.

But I do think Debian and friends do a disservice to linux by not installing the GNU documentation by default.

It's not about "doing a disservice", it's about obeying the licence terms.

The GNU Free Documentation Licence has (optional) clauses in it ("invariant sections") which preclude making modifications to certain parts of the documentation, which makes it effectively non-free. While some manuals licensed under the terms of the GFDL are entirely free, several of the major GNU projects have large political rants embedded within them, which can't be altered.

I can see both points of view here. If you write opinion pieces you might not want your opinion altering and republishing without your consent. However, I would also argue that a free software technical manual might not be the best place to put such opinion pieces, because there is a certain hypocritical aspect to championing free software rights, but not applying the same principles to the documentation of the same. It would not be difficult to keep the twain separate.

Full disclosure: I voted against keeping GFDL documentation with invariant sections in the Debian archive back when I was a Debian developer. This is because one of the primary benefits of free software is that everyone has the same rights and responsibilities as everyone else. Distributors and end users have the same rights to distribute and modify as do the original maintainers (albeit they can't relicense under different terms). The GFDL invariant sections make one organisation or individual "more equal" than others, and that's against the entire spirit of what free software is all about. This is one instance where I think RMS really dropped the ball. I might not agree with all of his philosophy but it's usually well thought out, and in this case I think it's not well thought out at all.

Other distros like Trisquel ship that documentation and they only have free repos.

It all comes down to the definition of "free".

Debian has the Debian Free Software Guidelines, upon which the OSI Open Source Definition is based. If you read them, you'll see that the GFDL with invariant sections fails the clause regarding "modification and derived works". Because it explicitly restricts your ability to modify and distribute.

Trisquel may have a different interpretation. I understand they are closely aligned with the FSF, so may not agree with the Debian stance.

This is one point upon which I do think the FSF is entirely wrong. Putting invariant sections into the GFDL made the licence firstly overly complex, and secondly made it incompatible with the OSI and DFSG interpretation of freedom. All just so they could include immutable political content in their documentation. The fallout from this made it a very poor choice to use for documentation, and was a spectacular own goal. If you follow the licence to the letter, it means you can't embed source code examples without being required to also include the invariant sections (you aren't permitted to delete them). This makes the licence impractical to comply with for a lot of common use cases. Overall, it's simpler to ignore the licence entirely and simply use the same licence as used by the rest of a given project's codebase. I did use the GFDL in a few projects at the time of its creation, but no longer do so after getting a better understanding of it. It's a bad licence.

Debian and the FSF have battled over this years ago (https://people.debian.org/~srivasta/Position_Statement.xhtml). It's as much the fault of Debian as it is the fault of the FSF to update the GFDL terms (which is possible, as many works are licensed as "current version of the GFDL + later").

What texinfo manuals? Who sells them? Genuine question, never heard of this before.

The FSF is selling them[1], but it's just a printout of the same content as you get with the Texinfo pages.

[1] https://shop.fsf.org/collection/books-docs

selling texinfo manuals? I don't understand, has that happened?

AFAIU, the sale of printed manuals was one of the earliest income streams in the Free Software/open source community, and for GNU/FSF in particular. O'Reilly and maybe some of its contributing authors (some of whom were project maintainers or contributors) made appreciable money selling printed documentation. But calling any of those examples a "business model" for open source development is a stretch.

The sale of support, OTOH, absolutely was an early business model. See, e.g., Cygnus Solutions, Red Hat, etc. And today many project maintainers make money in a similar manner, through contracts or outright full-time employment related to feature development and integration. But this happens just as often in BSD land as Linux land.

Then I'm failing to understand why not selling texinfo printed manuals is what allegedly makes OpenBSD documentation 'better'.

I think the insinuation is that the electronic documentation distributed with GNU- and Linux-related software was (and remains) subpar because of some early incentive to sell printed manuals, which would be more comprehensive and easier to use; whereas OpenBSD was never financially motivated to cripple their electronic documentation.

The GNU project does sell print copies of their texinfo manuals. But you could also download the manual and print it yourself. Buying the print is an optional way to support the development of the software.

what biz model?

Reply on news.ycombinator.com