One of the three OpenBSD users | Hacker News Reader

One of the three OpenBSD users

blog.tintagel.pl

54 pointsfcambus21 comments

Such a bug didn't deserve the snarkiness it got; as the article author notes, Linux itself remains a minor platform for end users, and benefits from people doing ports to Linux of software that has more users on other platforms. (For instance, Firefox and Chrome both have far more users on Windows than Linux.) But apart from the unnecessarily snarky response, I think this played out exactly as it should have. A kinder response would have been "none of us run OpenBSD, but if you can provide a non-invasive patch, we'll merge it". Which ended up happening anyway.

This is the way good things get done. New ideas should come from anywhere they can and the best ones bubble up into the product. Always reset expectations if anything that does get said ends up being misread. Let's face it; text is a horrible medium for communicating complex ideas. Open the communication channel accepting things at face value.

> Linux itself remains a minor platform for end users

What the fuck? Android is one of the most popular operating systems in the world.

That's a bit like saying that all Xbox users should count towards Windows numbers.

When people say linux, they don't care what kernel they're running - they mean GNU/Linux that has a reasonably close interface to that of a typical UNIX machine. I agree with Stallman &co. that the name "Linux" is imprecise, but you shouldn't pretend you don't understand what people mean.

Android systems don't run normal Linux binaries; Android has an incompatible ABI, not least of which because it puts the dynamic linker in a different location (and compiles that different location into every single binary). It shares the same kernel, but with a completely different userspace, and typically requires a separate effort to port.

(I'd have said "GNU/Linux" to avoid the ambiguity if I didn't think it'd massively and pointlessly derail the conversation. In particular, given the nature of this thread, it seemed needlessly adversarial.)

Google patches the snot out of the Linux kernel in Android. In fact in now contains a fair amount of ideas from OpenBSD. Theo De Raadt talks about some of the in his Pledge talk: https://www.youtube.com/watch?v=F_7S1eqKsFk

More importantly, why doesn't OpenBSD have procfs? Is there something fundamentally wrong with the idea? Is there some more standard alternative that portable software should be using?

Trying to answer this I found this old thread:

http://marc.info/?t=125341144400003&r=1&w=2

Arguments I see here:

* it's old code and not maintained.

* the interface has inherent race conditions (see Theo's comment about reading from procfs one byte at a time with sleep calls in between - I presume with the values changing underneath you)

Unrelated to this thread I found googling that OpenBSD and NetBSD have had a small but nonzero number of security issues in procfs over the years.

It wasn't long ago that Linux users fancied themselves the underdog being sidelined in favor of Windows. Now, the Linux users have become the Goliath flaunting their weight and key spokespeople even encouraging the rejection of all other platforms. It is a community defined by myopia, arrogance and a refusal to look outside its borders.

Linux has many different communities, not all of whom share the same opinions (or level of snarkiness). Some Linux developers may feel as you suggest, or more charitably, perhaps they just want to conserve their limited development resources focusing on providing the most benefit to the most users. But I don't think that view has become universal or even a majority, and I still regularly see projects merging support for other platforms, especially other Free Software platforms. I've merged such patches myself, written portability libraries, and even gone out of my way to test software I've written on GNU/Hurd. I've also seen ideas from such platforms influence Linux, and vice versa.

I certainly hope nobody views Linux and its community as consisting entirely of the kinds of people whose snarky remarks make the news. We could do with far less of those people.

In my more idealistic moments, I wonder how much better we could do if we focused on making one kernel better rather than half a dozen. Then again, in those same moments, I also wonder how much better we could do if we focused on making one desktop environment or browser better, too. None of those seem likely to happen.

While what you are saying is very true, recent developments around systemd seem to have changed a few people to target linux-kernel based systems than the BSDs.

As the autoconf files suggest, a lot of GNU software was written in the most general way possible, targeting even the macos/osx/windows/aix/hpux etc.

In this age of easy virtual machine installations, it seems odd that software authors of popular software are not targeting at least the BSDs.

> As the autoconf files suggest, a lot of GNU software was written in the most general way possible, targeting even the macos/osx/windows/aix/hpux etc.

GNU software originally ran on proprietary UNIX systems by necessity (because no non-proprietary UNIX systems existed), hence their portability; even after Free Software systems existed, having such software available on proprietary UNIX systems provided a great advertisement for GNU (especially since the GNU software typically worked better, and users could obtain tools for free with full source that they previously had to purchase separately in binary form).

More recent software (GNU and otherwise) doesn't typically include compatibility with obscure proprietary UNIX systems (especially long-dead ones), because such systems no longer particularly matter, and seem unlikely to gain new users. Portability to, for instance, HP-UX or Cray systems isn't even worth the time spent detecting and maintaining crazy build-time conditionals, or even the minor chance of introducing a bug. And, perhaps more importantly, the authors of the software don't particularly want to test on such systems, and a port you don't regularly test won't keep working.

Even a back-of-the-envelope order-of-magnitude estimate of the total amount of developer time, system time, and power consumed running ./configure scripts that autodetect conditions nobody seems likely to ever use again produces some scary numbers. ./configure takes an obscenely long time to run, often far longer than make.

> In this age of easy virtual machine installations, it seems odd that software authors of popular software are not targeting at least the BSDs.

That doesn't seem odd to me at all.

For most software, I wouldn't mind taking patches to support BSD, but that doesn't mean I'll go install a BSD system, try to figure out BSD conventions and policy, fix portability issues myself, and maintain continuous integration for BSD to make sure it keeps working. If I wanted to put time and effort into testing my software for portability to other systems, I'd first confirm that it built and ran on Fedora in addition to Debian; I've encountered issues with that due to Fedora's incompatible paths for 32-bit/64-bit libraries.

There are exceptions: I went out of my way to do exactly that as part of making sure XCB ran on the Hurd. I did that partly for novelty (I knew nothing about the Hurd, and afterward I knew slightly more than nothing), and partly because XCB needs to run everywhere X and Xlib can. Even then, though, the resulting port got maintained by actual Hurd users.

So while I'd take a patch to support BSD, I'd rely on the submitter of the patch to keep it working over time, and to let me know if it broke. (And if someone felt sufficiently dedicated to do so, I'd probably give them commit access to help maintain it.) But in the absence of a dedicated BSD developer interested in maintaining the port, I can definitely understand why people wouldn't go out of their way to target BSD.

The Unix community tried making one desktop environment better. It was called Motif, which mutated into CDE. Pushing monocultures leads to bland and a lack of creation, because you end up pissing off end users, like what happens when MS changes the start menu functionality, for example.

One kernel is even worse. OS research is still very young, and different types of operations need very different sorts of kernels. For an example based on the original article, getting information from /proc requires three system calls. You have to open the pseudo-file, and in the process allocate memory for a file descriptor, you have to make another system call to get the information, and then a third to close the file after you're done. That's six jumps in and out of kernel space just for one operation. For some tasks, especially when you're starting to get into hard realtime systems, that can lead to unacceptable amounts of delay. After all, would you be satisfied if the one kernel people settled around was the windows kernel?

> One kernel is even worse. OS research is still very young, and different types of operations need very different sorts of kernels. For an example based on the original article, getting information from /proc requires three system calls. You have to open the pseudo-file, and in the process allocate memory for a file descriptor, you have to make another system call to get the information, and then a third to close the file after you're done. That's six jumps in and out of kernel space just for one operation.

As it turns out, I'm working on a solution for that exact problem, within the Linux kernel. I don't see any particular obstacles to doing so.

As far as I know, Linux and BSD don't drastically differ from an "OS research" perspective. People doing OS kernel research that needs a "very different sort of kernel" tend to build a custom kernel, or run on L4 or similar. And sometimes that research makes its way into production kernels.

> After all, would you be satisfied if the one kernel people settled around was the windows kernel?

No, because it isn't Free Software, and nobody outside of Microsoft can change it.

> As it turns out, I'm working on a solution for that exact problem, within the Linux kernel. I don't see any particular obstacles to doing so.

oooh. I'd be really interested to learn about this... what exactly are you doing? (What can you do?!)

Identifying processes with file descriptors instead of PIDs, initially by adding a new flag to clone, CLONE_FD, that gives you back a file descriptor for the new process. The descriptor becomes readable when the process exits, so you can manage a child process and get its status without SIGCHLD. That allows a library to manage child processes without messing with the signal handlers of the parent process using the library.

Since the file descriptor holds a reference to the kernel data structure for a process (task_struct), that makes it race-free, unlike a PID (for anyone other than the parent process), and thus unlike a path within /proc (which uses the PID). So after getting CLONE_FD in, and adding ways to use a file descriptor in place of a PID in other syscalls, I'd like to add ways to use that process file descriptor to get back information about the process instead of using /proc (such as via ioctls).

While there's sadly no video available, I gave a presentation at Linux Plumbers Conference / LinuxCon that included discussion of CLONE_FD. See the slides at https://events.linuxfoundation.org/sites/events/files/slides... .

So, similar to pdfork in Capsicum and FreeBSD?

Similar, and we specifically took pdfork into account by trying to make sure interface-compatible userspace wrappers for pdfork could exist using CLONE_FD. (For instance, we made poll() on a CLONE_FD file descriptor set POLLHUP in addition to POLLIN, since poll() on a pdfork file descriptor sets POLLHUP.)

However, last I looked at pdfork, it didn't solve the problems we needed it to:

1) Most critically, processes created with pdfork still get picked up by standard waitpid/wait4 calls when they exit, which means you can't separate them from a global wait loop in the parent process. That makes it impossible to handle the child process from within a library without interfering with the parent process at all. (Similar to the problem that a process can only have a single process-wide SIGCHLD handler; those two problems together motivated the creation of CLONE_FD in the first place.)

2) It requires special calls like pdwait4 rather than just a simple read.

3) It defaults to sending SIGKILL when closing the fd, and with capsicum enabled it only allows that mode.

4) pdwait4 didn't actually exist in a functional form yet, so to work with current BSD you'd have to call pdgetpid and wait4 instead. (Which you can't do in Capsicum mode, or from a process other than the parent process.) This may have been fixed since the last time we looked at it, though.

5) pdfork only provides the semantics of fork, rather than the full semantics of clone.

All that said, I'm a big fan of Capsicum and the way it rethinks core APIs and resources to function around file descriptor handles.

You're not going to win people to your cause by responding to overgeneralisation with overgeneralisation.

what happened to syncthing usage since october? it seems the numbers have been dropping since the 0.12 upgrade...

Reply on news.ycombinator.com