Hacker News Reader

Comment by VWWHFSfQ | Hacker News Reader

It's my understanding that the microkernel architecture is slow. Nearly unusably slow. And that's why nobody uses it. Am I off-base? I'm interested!

View on HN

As more and more things move out of the kernel, the perceived performance problems of microkernels look less important. If you are doing your network protocols in user space, and your thread scheduling is in user space, and you're not using a traditional filesystem much, then suddenly nobody cares how fast the kernel is.

Yeah this, really. For the most part even monolithic kernels have kind of reversed trend in the last decade or so and there's a lot of push to move critical code out of the kernel. A lot of new kernel apis are built to avoid context switching, and part of that usually involves moving to a more asynchronous kind of communication between process and kernel.

Often these APIs even look an awful lot like late generation microkernel shared memory buffer protocols. DRI and uring in linux for example.

A lot of the "microkernels are inherently slow" meme is built on how earlier port-based message passing kernels like mach worked.

I'm really not understanding what you're saying here.

What the hell is a microkernel here? What kind of security are you talking about?

User space filesystem and network implementations still need access to the hardware. Multiplexing that access is a kernels job. The more you want to separate and hide that between clients the higher the cost.

As far as i understand your argument you are saying "If an application has a dedicated hard drive there will be little overhead"

People think a monolithic kernel can be faster because of high-level abstractions that make many calls within the kernel, for example if I write to a TCP socket and everything else is handled for me, in the kernel, by function calls only. People believe this is faster than having an isolated network stack that has to pass messages to an isolated network driver and all that.

But increasingly people realize that the performance of writing to a TCP socket in a unikernel also pretty much sucks and you get a much better result as you move more and more of it into user space. For example you decide, correctly, that TCP is obsolete and you switch to QUIC. Now the existence of the Linux TCP stack is of no value to you. You furthermore discover that Linux firewall, traffic control, and routing also kinda sucks, so you start using raw networking. Then you discover that trying to get your frames processed on the right core at the right moment isn't great in Linux, so you just take over the whole net device with DPDK.

Now, nothing in the whole Linux network stack is of any use to you at all.

The same thing can happen with storage. Maybe you started with files on XFS but then eventually you were using disaggregated storage where a service takes over the whole device with SPDK, and all the storage users are talking to the service instead of the kernel.

Of course, although you may be able to go very much faster, you also now took on all this extra responsibility, and so complexity, and so likely bugs.

Chances are the weird corner case where the TCP implementation blows up was seen in a dozen other Linux applications and fixed, but the same case inside your entirely userspace QUIC implementation gets to explode while the boss is doing the live $100M pitch demo. Awkward.

Code that is competing with Linux for performance is likely to be highly concurrent (a big risk factor for bugs) and doing fancy gymnastics with memory (another risk factor) so we're not talking about a landscape where bugs are rare or one where they're very easy to track down.

I believe this sort of thing is one reason Linux is putting more emphasis on safety. As usual for HN this bug already has people saying both "Rust wouldn't catch this" and "actually Rust would catch this", but even if you're quite sure Rust isn't the answer a safer Linux by whatever means makes this argument better for Linux than it is today.

The QUIC implementation was written in Rust, so the live $100M pitch demo is safe.

> User space filesystem and network implementations still need access to the hardware.

No, they don't. Filesystems don't need access to the hardware, that part is handled by the block device driver. Network stacks (e.g., TCP/IP) also don't need access to the hardware, that part is handled by the network interface driver.

There's more myth than truth[0]. In the early days, they were slow. Mach, used in OSX, is a representative of those early days.

Liedtke's L4 proved that a performant microkernel is possible.

Later, SMP changed the scenario considerably, as all of a sudden the microkernel multiserver fits SMP like a globe, while monolithic kernels need the complexity of locks to handle it.

[0] https://news.ycombinator.com/item?id=10824382

> Mach, used in OSX, is a representative of those early days.

Mach, the microkernel, is not using in OS X and never has been.

Mach, the API, is used (mostly due to legacy reasons).

OS X has a Linux-style monolithic kernel (with legacy Mach APIs in userspace).

>Mach, the microkernel, is not using in OS X and never has been.

The version of mach present in XNU is derived from OSFMK, which derives code from UoU's Mach 4 kernel, and from CMU's Mach 3 kernel. It contains improvements related to threading and contexts. It also definitely exists as more than an API. The code that makes up mach is present and identifiable, and thus is "in" XNU. [1]

>Mach, the API, is used (mostly due to legacy reasons).

See above. Abstractions are exposed, and traps are present, but this is not an API clone, it is mach, albeit modified from the original form. It was specifically chosen for its forward thinking benefits. Mach was not the basis for any major Apple OS prior to OS X. There were no "legacy reasons" to motivate its adoption.

>OS X has a Linux-style monolithic kernel (with legacy Mach APIs in userspace).

You do tend to see different opinions on this. Some argue macOS is a hybrid kernel, because it combines elements from a microkernel (mach), a monolithic kernel (BSD), and IOKit. A single address space is shared by the components though, which leads many to call it monolithic. [2]

1. https://github.com/apple/darwin-xnu/tree/main/osfmk

2. https://flylib.com/books/en/3.126.1.67/1/

Additionally macOS is going further down this path by moving kernel extensions into userspace.

I thought Windows and OSX were closer to hybrid kernels, at least with respect to running a lot of their drivers in userspace.

Almost every developer I have worked with considers NT and XNU to be hybrid at this point. Unfortunately, almost everything in the parent comment is incorrect.

> Unfortunately, almost everything in the parent comment is incorrect.

Avie Tevanian would disagree, but what does he know?

P.s. Nice job regurgitating Wikipedia in your other comment, without understanding it. I guess with only 30 karma, you've got to get your HN points somehow… Next time, read the Wikipedia entry on Mach so you'll learn that Mach didn't even become an actual microkernel until version 3.0. OS X's XNU kernel branched at 2.5. Oops.

The Nintendo Switch operating system, Horizon/NX, is a capability-based microkernel architecture. It's nowhere near unusably slow. In fact, its performance are pretty good. It does "cheat" a bit in that the whole OS runs on a single core, dedicating the remaining three cores to the game. But that may actually be the most efficient use of the hardware.

Yes, it’s going to be slower

All of those security checks between components and memory passing will cause it to be slower.

But that doesn’t mean it’s a worthy trade off.

People write software in python despite it being slower than C++.

Except that anything that is actually performance critical is written as a C extension python module.

I think that low level filesystem operations are very performance critical.

I doubt that mounting a filesystem is performance-critical. You could afford to fork an unprivileged user-space process written in Perl to parse these mount options and that would be fast enough for everyone.

Tell that to dockerd mounting images layer by layer , with k8s doing all kinds of emptyDir/PVC mounts on top. Pod start up speeds are abysmal now, they would be positively glacial with userspafe permissions validations

?? The layering unionfs (really aufs or overlayfs) mounts docker does are all in kernel. There's no userspace involvement in the actual file ops. I'm not at all sure what you think any of that has to do with the idea of usermode filesystem handling.

Anyways, on linux userland filesystem stuff (ie. fuse) is slow because the kernel manages the vfs layer and has to mediate. In a system designed for non-privileged vfs it doesn't have to be anywhere near so involved. And beyond that, there's really nothing in particular that a filesystem driver does that needs to be privileged to 'run fast'.

VFS is pretty much the posterchild case for how you can pull things out of a kernel and maintain performance, tbh. There's no reason at all it can't be fast except the kernel getting in the way rather than helping.

That's exactly my point. People who are doing lots of mounts are already demonstrably not performance-sensitive to a difference of a few milliseconds. They already waited 20 minutes for the stupid cluster autoscaler to provision a machine for them. They DGAF.

They just need some time off for leisure activities.

https://xkcd.com/303/

Generally yes. The microkernel architecture is theoretically better in many ways and this has led to many attempts over the years to switch to that architecture. Some attempts have been relatively successful. Over time, as hardware becomes more heterogeneous and more capable, we are likely going to see a move to microkernels.

Like many engineering decisions, the choice is not always clear cut. Other commenters have suggested microkernels are clearly superior and not choosing them is stupid, but there are many facets to that decision.

QNX is a microkernel and is used everywhere on embedded space, and it isn't the only one.

Microkernels are the preferred choice in high integrity computing.

This depends on the definition of the word microkernel, in the classical definition yes it will be much slower due to IPC, for something like an exokernel though it will be much faster than even a monolithic kernel.

Reply on news.ycombinator.com