What are the advantages of the Hurd over Linux/BSD? (2013) | Hacker News Reader

What are the advantages of the Hurd over Linux/BSD? (2013)

gnu.org

81 pointspmoriarty41 comments

> These were just examples, Linux is trying to catch up in ugly ways indeed ... In the Hurd, it's that way from the ground and there is no limitation like having to be root or ask for root to add magic lines, etc.

HURD might have user filesystem servers, but they're nowhere near Plan9's "each process gets a mount table, and then things like environment variables are just exposed process-local mounted filesystems."

Interestingly, given all the cgroups work put into the Linux kernel, you can almost achieve this postmodern "every process gets its own VFS, and uses it as scratch-space" effect with the one-two punch of:

    fusermount ... /mnt/env/1
    docker run -v "/mnt/env/1:/env" ubuntu

This, admittedly, still takes root-esque privileges somewhere along the line (either fusermount is SUID and run on the host, or the container must be run -privileged so you can mount inside it), but this isn't a fundamental restriction, just a historical artifact of Docker being designed before user-namespaces were finished. It's perfectly possible with raw unshare(2) to spawn a process as an unprivileged user, in which the process is acting with alternate-namespace UID0 privileges, and can mount whatever filesystems it likes in its alternate-namespace VFS.

The Linux containers people were directly influenced by Plan 9 in the design.

The need for privilege escalation to create the namespace is basically the rub for this sort of thing.

It seems like the main thing that makes true plan9-style namespacing unrealistic is the security implications, since the filesystem is used extensively to control privilege escalation. Whether it's through setuid flags or through /etc/sudoers or /etc/groups, at some point a privileged program (su or sudo) might be running in an unknown or untrusted namespace where the facts visible to that program make a solid case for allowing the escalation.

I'm not sure this tension can ever really be resolved without also adopting a plan9-style out-of-process escalation mechanism. Which is a huge departure from POSIX in and of itself, as far as I know.

I'm really not sure how HURD deals with this, to be honest, or if it does at all.

Actually there's just one /env in Plan 9, that behaves differently depending on what pid is reading it. So it's like /dev/tty in Unix.

Catching up to what? A 24 year old kernel with no userbase and uses IRC logs as the majority of documentation? Is Hurd useful as anything other than a research project? I would be interested to see what people here who have used Hurd have to say.

> Is Hurd useful as anything other than a research project?

Bear in mind that a research project can be interesting in its own right, not everything has to be about large scale use.

Now if HURD is an interesting research, that's an entirely different subject...

Even if it's not, not being interesting after so many years is, itself, interesting. I find the psychology of the attachment to software platforms fascinating.

At this point? None. It's a silly exercise on not going anywhere.

HURD uses Mach, which is a mid-90s academic microkernel that's not so good in practice. Particularly, it's so slow at IPC (which is critical to performance in a pure microkernel system) that those using it (Darwin/OSX/IOS, HURD) had to compromise and use a hybrid architecture: Running drivers and some other components of the system with kernel privileges; A popular design choice in the 90s (also BeOS, Windows NT) due to the immaturity of microkernels.

In late 90s/early 2000s, L4 happened. It went to great lengths to actually make achieving performance with a pure microkernel architecture possible. Afterwards, there were a lot of followup microkernels implementing the L4 interface, so these days when we say L4 we generally mean the interface, or any microkernel that implements it.

The HURD was watching, and sometime early 2000s they realized that the hybrid architecture was dead and they needed to move on if they wanted to stay relevant. There was a serious attempt at porting HURD to L4. It was already working, but the people behind it became disillusioned with the HURD, after realizing flaws on the architecture. I recommend reading the papers the L4 port people wrote on this. Back then, there were some ten to twenty HURD developers active.

After that, the HURD should have rethought its architecture and moved on with L4. Instead, they didn't continue the L4 effort nor fix the architecture. What they did was abandon it and start an entirely new port to a different microkernel, Coyotos, which didn't bear fruit, either. Throughout all this, the HURD was losing developers as they became disinterested.

Fast forward to 2005, Andrew Tanenbaum and his students released the first version (a mere skeleton, without even virtual memory support) of Minix3, and continued working from there http://wiki.minix3.org/MinixReleases , going a long way and bringing us to its current state. Right now, Minix3 has 20-30 active developers any given day, a few of which are working on it full-time, supported by funds coming from European Union research programs, as the reliability aspect http://www.minix3.org/other/reliability.html to Minix3 has been deemed important.

Meanwhile, the HURD has 0-3 active developers depending on the day, none of which working full time, and with no roadmap or organization whatsoever. Last I've heard, they introduced userspace driver support, which is a step in the right direction, but a bit silly as the real problem (they're still using Mach) is yet to be solved.

Minix3 next version, 3.3.0, is weeks away. Here's some insider info: It breaks ABI to adopt NetBSD types, skyrocketing compatibility with pkgsrc software. The system will for the first time be built dynamic, as mmap() is finally working and the dynamic library support has been adapted to actually do shared libraries using it.

As lack of proper dynamic library support was holding back X (which already ran, but with barely any software for it), I expect we'll have a lot of WMs, DEs and GUI programs from here on, and interest will pick up.

If you ask me about the HURD, I'd say it's not worth continuing. Just take along whatever salvageable ideas and concepts and move on. Working on a system that isn't at a dead end is one suggestion. Escape https://github.com/Nils-TUD/Escape , HelenOS and Minix3 are three such systems. Genode is not exactly a system but it is interesting in its own way, and Plan9Front http://ninetimes.cat-v.org/ is very interesting even though it is not currently a pure microkernel system (it can be made so without breaking anything thanks to the awesome design). All these systems I suggested are Free Software, interesting, promising and actually active.

Another intersting microkernel might be seL4, which is an L4 implementation (as you might have guesses from the name) focussed on security that will be released as open source (code and correctness proofs, as all good security software should be in an ideal world) at the end of July: http://sel4.systems/

For a gallery of L4 microkernels, just look at Genode. They support a bunch and also some non-L4 yet interesting ones: http://genode.org

Didn't know seL4 was going to be open source. Thanks.

Here I was thinking: Wouldn't it be nice if seL4 were Open source.

I remember when I first realized that the dream of GNU+Mach wasn't moving very quickly. That was 1991.

With hardware support for virtualization I see the OS landscape going a different direction, it seems the Holy Grail of "everything is userspace" will be effectively side-stepped by "everything is sandboxed".

Here's something that's even more "out there": an experimental OS and programming environment where all protection is done statically, at compile time. You can do things as crazy as run all "user" processes and the kernel in the same (kernel-mode) address space safely, because the system compiler enforces type-safety and memory-safety. It's pretty cool! (my memory is a bit weak on this paper so some of the details might be off, but it's well worth the read)

http://research.microsoft.com/apps/pubs/?id=69431

Burroughs did this on the old 5000, in the 1970s I believe.

Sounds interesting if you have very constrained hardware, but you'd have to trust the compilers or you wouldn't ever install 3rd party binaries.

And I think gaius is right. Burroughs did it before.

Presenting services in user space has always been a benefit of microkernels, which is welcomed considering that I frequently make use of FUSE or OpenVPN.

The only concern for me is performance. Though I believe the service server provided by the Hurd would outperform FUSE, I'm not sure that it is worth the overhead of all other parts in the entire system. If the performance decrease is negligible, I'll be happy to use the kernel.

The performance history is old news.

QNX, Symbian, L4, PikeOS are just some examples of operating systems with micro-kernels that achieved success on the market.

Only the desktop/server OSs still lag behind due to compatibility with existing software.

Have they worked out how to improve performance, or have they succeeded in spite of it?

I run software that (thanks to poor design, IMO) manages to bottleneck context switching on Linux's monolithic kernel. I can't imagine this not being worse with more of the OS pushed out into userspace, though I'd love to be proven wrong!

Kernels in the L4 [1] family are effectively designed to reduce the overhead of calls across process boundaries, to the extent that software is unlikely to behave any differently from a kernel like Linux.

It can potentially be even better, since you can potentially get more direct access to whatever resource you're contending on.

1. http://en.wikipedia.org/wiki/L4_microkernel_family

According to this paper[1] about L4, it seems to be the former (that is, performance has genuinely improved).

[1]http://www.nicta.com.au/pub?doc=6930

L4 was a big push to improve microkernel performance. It's improved to the point that it only costs a few percent throughput, which means context switching itself on L4 is much faster than on Linux.

The Mill CPU design doesn't have a user/kernel mode split, and it's also single-address-space (still with the same level of protection, just a different mechanism). Protection domain switching is done through portal calls, which are essentially shared library-like indirect calls with metadata for the CPU. Further, saving and restoring things like registers is done asynchronously. That should remove any remaining performance difference (and make monolithic designs faster too).

Where can I find more structured info about the mill cpu? Their www is basically empty.

The vast majority of information available right now is in their videos: http://millcomputing.com/docs/

Their forum has some good responses from the team on follow up questions asked by the community as well.

except the hurd runs on mach... i read a bunch of the hurd documentation a couple of years ago, including the paper by the people who tried porting it to L4.

i also tried installing the debian port on an old desktop but had some driver issues. i guess most people run it in a hypervisor.

there are a lot of architectural advantages to putting all drivers in user space. the hurd isn't really totally comparable to plan 9 (because it runs on a microkernel) or minix (because it's thought of as being independent of the any partciular microkernel). but it seems like the community/political issues -- plus the amount of cruft that's bound to accumulate in a codebase that old -- has nearly slowed to project to a grinding halt, which is a kind of a shame.

Here's Andy Tannenbaum's response to the performance issues of microkernels: https://www.youtube.com/watch?v=bx3KuE7UjGA#t=2832 (should start at about 47:15)

Tl;dr: He thinks reliability trumps performance. I mean, he has a point, but then there's the reliability of the hardware, and any process can be made more reliable if you know the details of the the software instead of just trying to make everything about the system the software runs on reliable. I guess a reliable OS can be a middle ground between bespoke systems with redundant hardware and a single computer running Debian. Well, I guess a reliable OS is always a good thing, but it'll never be enough for everything.

I am with him. It was putting performance above reliability that has brought the industry to the current state of security exploits that cost millions in tooling, bug fixes and paying damages.

We are still very far from other industries where reliability is taken seriously.

Reliability and security are different things. Industrial controller manufacturers often don't give a shit about security, but their products are much more reliable than x86 running Linux.

I think otherwise. Reliability implies security.

This argument cuts both ways. The emphasis on performance and, you know, working is what led to the massive amount of money in the market.

That is, yes there was a cost to this. However, leaving aside the unknowns of whether there would still be millions in tooling necessary for security even with microkernels, you have to provide some sort of reasonable argument that the current market (> billions) would still exist. You can't just take that for granted.

As I heard once at one software engineering talk, IT is the only field where one can buy a pair of shoes that explode randomly when tying the shoe laces and everyone will find it normal.

Which, of course, completely ignores all of the bat shit insane things people used* to sell and do to each other.

The difference isn't that IT has some dangerous crap in it. The difference is that you know some of these dangers. And, you feel that it doesn't belong for some reason.

Consider, you probably have highly charged electrical wiring to your house. Fairly high pressure water. Possibly gas. Literally explosive. Several drawers of sharp instruments in your kitchen. Possibly power tools that are literally spinning blades. Of sizes from less than a centimeter to upwards of a meter.

Think of a computer less as a safe device that only gives good results, and more as an extremely powerful tool that does what you tell it to do. Very fast. And then be thankful that it won't take your fingers off if you slip while pushing a board across it.

* Who am I kidding? We still do a fair bit of it. :)

Also... apropos of what? I'm not sure how this followed from my parent post.

In general, a kernel that requires a five transitions (user->kernel->user->kernel->user) for system service calls, will be less performant than one which requires only three (user->kernel->user) The impact is magnified with multiple services being involved. So if you naively design your disk driver as a user level process, your file system as a user level process, then something like read() from a user's applications goes: read -> kernel -> user(fs) -> kernel -> user(disk) -> kernel -> user(fs) -> kernel -> read(app). It was really really slow when things had different page mappings but it gets better if you have an alloc area where everyone shares the same virtual address space (keeps TLB thrashing down).

Per another comment on this thread the Docker as P9 process is pretty close. But it doesn't share services per se across users in the space. Spring (Sun's research OS) looked at cutting the penalties down for moving kernel/user with some nice 'doors' kinds of things. At the time there was discussion that with enough address space (so that you're VM system got the benefit of warm caches and no page remapping) you could really do some interesting things. Were I retired I would probably spend some time playing with that stuff on a modern 64 bit architecture :-)

  five transitions (user->kernel->user->kernel->user)
  three (user->kernel->user)

Aren't these 4 vs. 2 transitions?

You are correct 5 contexts, four transitions.

> In general, a kernel that requires a five transitions

I wonder if it couldn't be accelerated with hardware support...

Or with libguestfs (no root needed):

    guestfish -N test1.img=fs
    guestmount -a test1.img -m /dev/sda1 /tmp/mnt

Here's a more interesting list of potential file system translators:

https://www.gnu.org/software/hurd/hurd/translator/wishlist.h...

the best part of hurd is that systemd does not support it :)

Reply on news.ycombinator.com