Hacker News Reader

Comment by nullindividual | Hacker News Reader

The NT kernel and XNU (macOS) are considered hybrid kernels. QNX, a microkernel, is absolutely everywhere (i.e., cars).

Performance of microkernels is or was a hotly debated topic back in the '90s; microkernels generally had lower performance than their monolithic counterparts. There was a famous argument comparing Linux (monolithic) to MINIX (micro) [0]. Wikipedia has simple explanations for the differences between hybrid, mono, and micro kernels that serve as a decent primer [1] and a visual that I find helpful [2].

[0] https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_deb...

[1] https://en.wikipedia.org/wiki/Kernel_(operating_system)

[2] https://en.wikipedia.org/wiki/Hybrid_kernel#/media/File:OS-s...

View on HN

Not to mention Minix running on ~every recent Intel system: https://en.wikipedia.org/wiki/Intel_Management_Engine

Microkernels still have lower performance than their monolithic counterparts. It makes sense really: the major difference between the two is protection barriers between components. All else being equal, a system without those barriers is faster than one with them unless crossing those barriers has zero cost -- which it only really does in the case of proof-carrying code and similar software solutions, where the "barrier" is more of a guarantee that the subsystem behaves as it should.

A lot of microkernel research in the past focused on the "all else being equal" aspect, attempting to come up with innovative ways to reduce protection domain crossings in microkernels. For example, mapping graphics card memory or network device buffers directly into the address space of the client means that the hardware can deliver data directly to the client. However, monolithic kernels can also benefit from these sorts of innovations!

As to why consumer OSes are all monoliths (yes, this includes Darwin and NT -- "hybrid" is a marketing term as all the important stuff runs in a single kernel address space; see https://en.wikipedia.org/wiki/Hybrid_kernel): in consumer OSes, the 5% (or lower) performance hit of a microkernel is a big deal. In other spaces, such as embedded, the benefits of microkernels (reliability, security guarantees) are sometimes more important than raw performance.

> Microkernels still have lower performance than their monolithic counterparts

This is what I've understood, that the idea sounds good but in practice isn't as good as it seems like it would be.

You necessarily cross from userland (end program) to OS to userland (driver) for most tasks that monokernels just do with the first userspace->OS transition.

There have been architectures specifically designed to support some form of IPC or other directly between processes without going through the kernel, and thus without having more overhead. But those are a bit esoteric.

The Mill is one, that has been in development for a while, but its company hasn't published much lately and its current status is a bit unclear. It would have had something very similar to Spring/Solaris "Doors" in hardware, where the caller could have passed temporary permissions to memory ranges. A "kernel" would have been a process like any other, just with more permissions.

Not to mention Minix running on ~every recent Intel system

And L4 on every Apple device with a secure enclave: https://support.apple.com/nl-nl/guide/security/sec59b0b31ff/....

>Not to mention Minix running on ~every recent Intel system: https://en.wikipedia.org/wiki/Intel_Management_Engine

But not as the actual system OS. Minix is doing a very little, running on a little side processor that also does very little. It's not running on the main CPU, nor having to interact with the insane number of possible peripherals that real CPUs and OSes have to interact with, nor with real system loads, nor with many of the other constraints that real OSes have to deal with.

Yes, it's there. Yes, it's there in really massive numbers. But, compared to what the OSes are doing on the main CPU, Minix is doing a toy job. So I'm not sure that it counts as evidence for much of anything.

It's simply not true that microkernels have lower performance. At least it can strongly depend on what you measure. There are tradeoffs, but high-level workflows can accomplish comparable performance. The problem is that sometimes requires rearchitecting software against alternative interfaces. It's becoming increasingly more common for very high performance applications to all directly to hardware and bypass the kernel altogether. Async io such as io_uring also further reduces disadvantages of a microkernel, especially on systems with many cores.

Shouldn't the microkernel architecture be capable of having fewer barriers in the fast path? e.g. you could have hardware like a NIC or GPU be capable of partitioning address spaces to have per-program command/response queues. Then the kernel asks on an administrative queue for a program's IO queue to be created, configures the memory controller (assuming an IOMMU) for the program to have those queues in its address space, and context switches to the program. While the program runs, there are zero barriers and it can simply talk to the hardware directly by writing into pcie space. Hardware knows what queues commands are coming from and can enforce whatever resource partitioning within firmware.

My understanding is that (perhaps except for virtualization/address space partitioning on the hardware device to be able to use it with many programs at once) this is approximately how/why user-mode networking is used for high performance today?

Monolithic kernels also have these fast paths. It's how GPU drivers work under Linux - your process submits command buffers directly to the GPU.

That all requires a lot of hardware support that probably didn't exist at the time, and is probably still esoteric today.

This is the basic idea behind the exokernel concept: have the kernel's only job be to enforce protection boundaries between processes, and then expose an interface to userspace that is identical to what the hardware exposes (eg. directly write to buffers and queues):

https://en.wikipedia.org/wiki/Exokernel

No production exokernels were built, but the concept lives on as virtualization. Most hypervisors (eg. Xen) work on a substantially similar principal: the hypervisor is responsible for securely multiplexing the hardware between guest OSes, and then the guest OS talks to an interface that basically looks like the hardware.

A step beyond this is to get rid of the guest OS and have application code talk directly to the hardware interface. This is a unikernel (or called a "libOS" in the original exokernel papers), and as you predict, unikernels running on a hypervisor are often faster than traditional monolithic kernel OSes. The main thing blocking their adoption is that they require significant rewrites to application-level code: most applications (and the libraries they depend on) generally assume POSIX syscalls are available and would need to be largely rewritten to work in terms of direct hardware access. You can rewrite POSIX as a library in userspace (this was the approach taken by the original exokernel research), but most of the performance gains come from bypassing inappropriate abstractions, so if you use the same abstractions you get basically the same performance.

https://en.wikipedia.org/wiki/Unikernel

Whatever you want to call it, NT is neither a pure microkernel nor a pure monolithic kernel. Linus' opinion of the term 'hybrid' being a marketing term is great and all, but it's just an opinion.

You could call NT a 'modified microkernel implementing monolithic kernel concepts', if you wish. The word 'hybrid' is much shorter, though.

  in consumer OSes, the 5% (or lower) performance hit of a microkernel is a big deal

This does not feel like it should have been true for a long time. Computers are just stupidly fast now. In the 90s, sure, performance was way more important to prioritize, but I would love to see the needle swing the other way to getting a more secure and robust OS. My system already squanders so many resources on Electron JavaScript nonsense. Why not let the OS take up a few more cycles to give me hard security guarantees.

Somebody tell that to my iPad which goes out to lunch for 30 seconds when I am browsing the web and always seems to be listening to the music in its head. Or the XBOX that takes 30 seconds to sorta kind boot but really 3 minutes for all the menus to load.

You can do that and people already do. The most microkernelly consumer OS is macOS/iOS and there you see a lot of code has been moved out of the kernel with time. Example: whilst APFS is in-kernel by default that isn't much security risk. More vulnerable components like FAT drivers have moved into user space. On iOS the networking stack also runs in user space for apps that use Apple's APIs!

https://devstreaming-cdn.apple.com/videos/wwdc/2017/707h2gkb...

So the whole TCP/IP stack is just a regular userspace library on that platform. Only the wifi driver runs in-kernel.

These platforms are made up of literally hundreds of Mach servers. Relatively little code runs in the kernel when you examine the structure of the whole OS. The kernel is really only running some hardware drivers where moving them to userspace is hard, and some basics like memory management, process scheduling, IPC and code signature verification. The kernel just isn't that big.

It's important to realize though that moving even more stuff to processes does not automatically improve security or availability. Imagine there's an RCE vuln in your root filesystem code. Being in a userspace server doesn't help you:

1. If you can control the FS you can make a root owned file appear anywhere and then just run it, or alter system configuration in any other way.

2. If you can crash the FS the core kernel can't easily recover without special cases because where would it restart the FS process from? If you say "initramfs" then you're back to having an FS in the kernel.

In practice there are lots of cases like that. Sometimes it's worth it - modern OS' can generally recover from a display server crash and run graphics mostly in user space. Sometimes it's not.

Finally, bear in mind that Linux is very insecure due to its huge feature set and reliance on C, but kernel exploits are often not the biggest security problem on other platforms. The most serious bugs are all in userspace already on most platforms.

There's no way a 5% performance difference is the reason. These OSes were designed in an era where you could get a 5% boost just by waiting a month.

I think the real reason is they were also designed at a time when security was not as advanced as it is now, especially for Linux, and microkernels are more secure but also harder to write.

The Tanenbaum-Torvalds debate Wikipedia page seems to be missing a sidebar listing the combatants, strength of each side, causalities, etc.

> # Intel Management Engine revelations

> In 2017, it was revealed that Intel was running MINIX inside the Management Engine, a separate processor incorporated in Intel processor chipsets since 2008. As a response to this, Tanenbaum wrote an open letter to Intel claiming MINIX to be "the most widely used computer operating system in the world".[17]

Hilarious.

I only wish I had a bucket of popcorn before I began reading this.

https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_deb...

There have been attempts at using QNX for a mainstream OS though, neither that gained major success but not because of lack of technical merit.

In the mid 1980's, the ICON microcomputer system, perhaps known mostly for being used in Canadian schools.

In the early '00s, QNX got jealous of Linux's success and started opening it up to try to attract app developers from the open source community.. but closed down the program before it could gain traction.

In the early '10s, it was bought by Blackberry and used for the OS of a new wave of Blackberry smartphones and a tablet.

BlackBerry 10 devices were even capable of running Android (kitkat?) apps, which is an amazing feat by QNX. I wonder how they did it

QNX is promoted as a microkernel, but I'm skeptical. Microkernel doesn't just mean that we have modules that have their own threads. It refers to actually moving kernel functionality outside of the protected kernel space: device drivers, file systems, protocol stacks and such are in userland.

QNX is everywhere - including space http://www.qnx.com/news/pr_1446_4.html?lang=en

FreeRTOS too

Reply on news.ycombinator.com