Maybe we should revisit microkernels
notes.hella.cheap
notes.hella.cheap
It was very efficient, a 200 MHz Nokia could run respectable loads. But the API was hell, which wasn't caused by the microkernel architecture, but rather by inheriting mid-90s EPOC idioms.
With a normal C++ stdlib, that would have been a very fine environment indeed.
> Better security, better reliability, better modularity. Better security because in a muicrokernel [sic] system a bug in one driver only gives an attacker access to that subsystem (or maybe even that driver), as opposed to having unlimited access to the whole system like is the case with mainstream operating systems today. Better relaibility [sic] becasue [sic] a crash in one subsystem only crashes that subsystem instead of crashing everything; if windows were a microkernel system then the crowdstrike bug would have just stopped some IT security staff from getting telemetry instead of being front page news. If linux were a microkernel system then the linux kernel team wouldn't be responsible for merging every driver for every hardware device, and wouldn't be responsible for vetting code that they would have to become experts on the inner workings of every chip supported by the linux kernel to properly vet.
Is false. In particular:
- A kernel-level exploit can almost always be escalated. If I control your filesystem, you're SOL
- A kernel-level crash in any subsystem brings down the whole system. If your file system crashes, or your GPU is in an unstable state, there is no coming back
- Modularity is a question of clear boundaries. Whether those are static linking, dynamic linking, or XML-RPC doesn't change the level of modularity.
And so on.
Basically, none of the upsides have panned out. On the other hand, code complexity explodes. Microservices need to know who to call how. The whole "a crash in one subsystem only crashes that subsystem instead of crashing everything" means every subsystem needs defensive code for things going wrong elsewhere. KLOCs go up, bugs/KLOC stay constant, and things get less stable.
Plus, it's hard to reason about systemically, and without excellent telemetry, neigh-impossible to debug.
There are parts that are so simple that it can be practically guaranteed that they are bug free, e.g. the keyboard driver. Also the microkernel itself can be practically bug-free.
A GPU driver is unlikely to ever be bug free, but if the GPU is in an unstable state, because the keyboard should still be working fine, it should be easy to reset the GPU and continue the work without rebooting the computer.
On Linux I have seen cases when the GPU crashed, but switching to a virtual console and back avoided the reboot, by reinitializing the GPU, but I have also seen crashes when not even the keyboard worked, which should never happen, so a reboot was the only solution.
With device drivers in separate processes, bugs in one should not corrupt the memory of others, so restarting a device driver should be enough, instead of rebooting the computer.
> Microservices need to know who to call how
Here there is no difference between a monolithic kernel and a microkernel. With a monolithic kernel, the applications need to know which syscalls to use.
Libraries like libc or liburing can be implemented either by using syscalls to a monolithic kernel or messages to the processes that own peripheral devices, without any change in the API used by application programs. Obtaining memory pages shared with device driver processes can be done by creating file mappings for some pseudo-files with standard names, which has no essential difference from doing I/O by opening device nodes from the /dev pseudo-filesystem in UNIX-like OSes.
Any complex program must be designed from the beginning to be easy to test and debug. If the device drivers are distinct processes, there is no reason to be more difficult to debug them. On the contrary, because the interactions between them are more limited than when they live in a common address space, they should be easier to debug.
A micro-kernel with the associated device driver processes would contain more or less the same code that is now included in the Linux kernel or in any other monolithic kernel but it would be separated in multiple parts with clear boundaries and interfaces between them.
This can only make much easier the reasoning about how the entire system works. The downside is that the interfaces must be well defined. The history of Linux shows a huge number of badly defined interfaces, which later had to be changed. Even in the syscalls, which are supposed to be stable, there are many syscalls already at their fourth or fifth variant, with the previous variants still kept around despite being obsolete, for backwards compatibility with legacy programs.
Nevertheless, defining good interfaces is not that difficult, especially today when there is much less pressure to save every bit in a parameter list (even when taking into account the current memory prices). Most Linux interfaces that had to be replaced were replaced for not having features that were obvious as necessary since the beginning, e.g. for not having a parameter with option-encoding flags or for using some data types that were too small.
And the major motivators for KMS/DRM, which moved a lot of that functionality into kernel, was performance and stability.
Yes, it should be. I can name zero times in the past half-decade that it's been possible to soft-reset the hardware in my GPU.
The "reset the GPU switch" is the power switch on my computer.
That's true of most hardware. If my ethernet, wifi, or whatever else enter an unstable state, on paper, the monolithic kernel can do the same thing as the microkernel: unload the module, then reload the module. In practice, I've virtually never seen a driver able to pick up a piece of hardware in an unstable state and be able to "reset" it.
> With device drivers in separate processes, bugs in one should not corrupt the memory of others, so restarting a device driver should be enough, instead of rebooting the computer.
Device drivers can corrupt other devices easily enough. The hardware itself would need to be built for isolation.
> Any complex program must be designed from the beginning to be easy to test and debug. If the device drivers are distinct processes, there is no reason to be more difficult to debug them. On the contrary, because the interactions between them are more limited than when they live in a common address space, they should be easier to debug.
That's exactly the myth.
In practice, a complex data flow for a heisenbug might start at the network layer, go through the disk, and land on the GPU. Tracing that on a microkernel requires instrumentation which, while possible in theory, I've never seen in practice.
I'd love to see a hardware stack where everything is ECC, devices are isolated, etc. but for now, on a typical computer, plugging in a bad USB device can brick the whole system (regardless of the software on it). The assumption is that devices on your PCIe bus can be trusted.
> This can only make much easier the reasoning about how the entire system works.
You make very strong statements, without support.
There's a reason Linux runs on every Android phone, almost every router, and increasingly, devices like home microwaves.
This was possible with old-school graphics cards in the nineties and early 00's, when they were mostly a framebuffer, with a bit of 2d acceleration.
What history? What proof? What evidence?
What commercial microkernel systems are you referencing that suffer from trivial LPEs? Or whole system DoS from unprivileged code execution? To the extent that you argue your alleged failure modes are fundamental.
> none of the upsides have panned out
Again, what history? What proof? What evidence?
One of the stated upsides relative to monolithic kernels is superior reliability. There are numerous microkernels reliable enough for usage in critical flight systems. None of the standard commercial monolithic kernels are even in the vicinity of that.
Another of the stated upsides is superior security. There are multiple microkernels secure enough to be platforms for systems that need to be secure against state actors. None of the standard commercial monolithic kernels are even in the vicinity of that.
History has proven the opposite of your claims. The concrete evidence is the opposite of your vague unsupported assertions.
...
> standard commercial monolithic kernels
Note the path from A to B.
Fully monolithic: Linux, FreeBSD, NetBSD, OpenBSD, Solaris, AIX, zOS.
Mostly monolithic: Windows, MacOS
To get to microkernels, we're reaching for QNX and Minix.
Monolithic kernels won *hard* with good reason.
> critical flight systems
Hard RTOS is a whole different domain of engineering. You're very confused if you think the same principles apply to something which makes sense in a mainstream OS.
You claimed that microkernels suffer from the same security problems that plague commercial monolithic kernels such as Linux, Windows, macOS, iOS, Android, etc.
You claimed without evidence, and I quote, “none of the upsides have panned out”.
In contrast, microkernels have completely dominated the field, displacing prior monolithic kernel solutions, in domains where reliability and security are not the lowest priority. All you have really supported is that the broader commercial IT landscape continues to prefer easily hacked systems as long as they can save a buck.
FYI: For most commercial network equipment (internet backbone and similar), reliability and security are the highest priority. Most of that is Linux, BSD, and friends.
That sentence is grammatical nonsense. Try again to articulate your point so it is coherent.
Furthermore, you have still failed to present any historical examples for literally any of your bullet points or articulate any of the claimed upsides you purport have failed to pan out. In contrast, I have pointed out two frequently claimed upsides, reliability and security, that have demonstrably succeeded to such a degree that prevailing monolithic systems were largely replaced with microkernel systems in domains where reliability and security are highly valued such as aerospace and defense. To the extent that monolithic systems are still used, they were either the legacy option that has not been replaced or chosen where cost reduction is the highest priority, not security and reliability.
> FYI: For most commercial network equipment (internet backbone and similar), reliability and security are the highest priority.
Frankly, I find it baffling that you think that nuclear weapon systems, military jet fighters (which are network connected by the way), and commercial jetliners (also network connected) have lower reliability and security requirements than commercial networking equipment. And this is ignoring the fact that commercial networking equipment is some of the worst with respect to reliability and security. Cisco, one of the largest vendors in the space, routinely has trivial remote authentication bypasses allowing complete takeover. Here is one from last month: https://app.opencve.io/cve/CVE-2026-20181. One from earlier this year: https://app.opencve.io/cve/CVE-2026-20079. Or this one: https://app.opencve.io/cve/CVE-2026-20127. How about this one: https://app.opencve.io/cve/CVE-2026-20182.
Commercial networking equipment security is a joke. Find me literally any technical person who would claim that Cisco can defend against state actors. I will wait.
Also very hard to debug
I'm hoping it can become my daily driver.
Eh, no, author mixed things up. To access a hardware device, you need ioperm, which is at least 20 years old. (Well, not today, but it was sufficient back then...). IOMMU is completely optional.
The microkernel overhead was too high because any operation (even the one that does not need access to hardware, like reading file from file cache) required a context switch. This does not change with IOMMU. And while having multiple cores helps, you are trading context switch to intra-core cache invalidation and extra scheduling overhead (because no modern system can just have a single core spinning forever).
It allows the operating system to allocate peripherals mapped in the special I/O address space to dedicated device driver processes. For example it could allocate the real-time clock to such a process and a serial UART interface to another process.
Most modern peripherals are memory-mapped. However, you are right that even on the older systems without an IOMMU, the operating system could allocate peripherals to separate device driver processes, using the standard CPU MMU and mapping the peripherals using appropriate page attributes, e.g. as uncacheable memory regions.
The IOMMU is not needed to protect the OS and the processes between themselves, but it is needed to protect both the OS and the processes from rogue peripherals. This is needed regardless if the kernel is monolithic or a microkernel.
The microkernel overhead in the old microkernels, like Mach and all those inspired by it, was indeed too high due to context switches, but that has nothing to do with microkernels but those microkernels just implemented an extremely inefficient method of IPC, which unfortunately has tarnished the reputation of microkernels.
The only right form of IPC does not use any context changes, i.e. it uses message queues located in shared memory pages, with zero copying and no context changes, which is much faster than the traditional syscalls used with monolithic kernels.
In TFA there is another thing with which I do not agree, it is said that message queues need atomic CAS (compare-and-swap, a.k.a. compare-and-exchange in Intel parlance).
In my opinion, using CAS for implementing message queues is a big mistake, because this instruction is non-deterministic and the execution time for any program that uses CAS is unbounded for the worst case.
One-to-one message queues do not need any atomic instructions, they just need store-release instructions and either wait-for-not-equal loops when the waiting times are expected to be short or futex_wait (or similar) syscalls when the waiting times are expected to be long.
Waiting for (multiple) events is one of the principal functions that must be implemented by a microkernel for enabling interprocess communication, together with a memory mapping API. IPC per se must not be implemented by the microkernel, but by user-space libraries.
The message queues that are many-to-one, one-to-many or many-to-many, can use the atomic fetch-and-add instruction (LOCK XADD on x86-64), to dynamically partition the queue, allowing the concurrent insertion and extraction of data into/from the queue by all communicating processes. In this case, only the updating of the queue pointers is serialized, resulting in very low overheads for the communication through the message queue, even in cases of high contention.
While I agree with you that TFA is incorrect in some details, I agree with the general idea that today one could implement microkernels that would be simultaneously faster and safer and easier to maintain than a monolithic kernel like Linux.
Nonetheless, the chances for such a microkernel to appear and to be successful are very low.
The reason is that the only way to be successful would be to implement a very easy transition to it from one of the popular OSes, e.g. from Linux.
This means that it would have to also implement a syscall interface completely compatible with that of Linux, to allow running unmodified legacy programs, and also a device driver API compatible to that of Linux, to allow the reuse of the existing device drivers.
Probably the easiest way to do this would be to start from Xen, still using the root Xen domain with a Linux kernel that manages the I/O, and running legacy Linux applications in virtual machines.
From this, another special domain with the new microkernel should be added, and then the peripheral handling should be migrated gradually from the Xen Dom0 Linux to the microkernel with its associated device driver processes, starting with those more important, i.e. time, keyboard & graphic pointer, GPU, NVMe, Ethernet, USB.
In parallel with migrating peripherals to the microkernel, user applications should be migrated gradually, to use the new I/O IPC instead of the Linux syscalls.
Only such a gradual evolution could succeed. Otherwise, a nice microkernel with which you cannot run any useful application would fail, like the many attempts of creating better operating systems that have failed in the past, even when they were backed by huge companies, like IBM OS/2.
I believe that the I/O API used by applications with such a microkernel should be extremely similar with that of liburing (Linux io_uring), but which would use internally IPC with the appropriate processes, instead of kernel syscalls.
Only Fuchsia might have a chance, because of the disproportionate control that Google has over the Android applications and because of the constraints imposed on those.
Probably the most widely deployed microkernel ever: it's inside every Intel CPU for the last ~20 years.
Already has a mostly-working port of the NetBSD userland.
It was shaping up quite well until Andy Tanenbaum retired. Since then, nothing.
Of course, Intel has never released any of its code, which is terrible -- but legal and license-compliant.
Concerning performance: studies suggest that Minix 3 is significantly slower (10 to > 100 times) than L4 and even Linux (see e.g. https://www.cscjournals.org/download/issuearchive/IJE/Volume... or https://www.minix3.org/theses/priescu_thesis_2012.pdf).
Concerning "inside every Intel CPU": we actually don't know which version that was supposed to be and how much it was modified.
All the I/O requests would be submitted in queues and whenever a queue would be empty the server process would sleep by invoking a wait for event syscall, like futex_wait of Linux. Obviously, when a device driver sleeps and must wake up, that involves context changes, but this will happen only when the peripheral is seldom used, so the performance loss is not important. Once the device driver process is active, no context changes are necessary for data transfers between processes. When there is heavy I/O activity, a few cores may be occupied with it, but they will always correspond to the active I/O subsystems, not to any that exist but are not currently in use. The bulk data transfers should be done by DMA directly from the user buffers, so the device driver processes should only orchestrate all the transfers, which should not consume much CPU time.
Nonetheless, I believe that the performance could be improved by reserving a single core for the microkernel, for hardware interrupt handling and also for the keyboard and graphic pointer drivers. Only in very big systems with hundreds of cores it might be necessary to reserve more than 1 core.
A user API like that of liburing of Linux can be reimplemented almost with no changes in a system with a micro-kernel. That means that a user application would have an AIO completion queue, which instead of receiving data from the Linux kernel would multiplex the completion messages from all the device driver processes.
For the opposite direction, from a user application to the device driver processes, the implementation of the I/O request submission queue would be a little more complex, because of security requirements. Processes that trust each other, e.g. the children of a single process, could inherit and use a common submission queue. Otherwise, the submission queues must be distinct, i.e. in mapped pages that are shared with the device driver processes, but not between the user processes. Because polling multiple queues might waste time in a device driver process, it is likely that submitting a batch of I/O requests should be done by writing the shared buffers with data, then invoking a syscall similar to io_uring_enter, which would alert the corresponding device driver processes (by posting the request in single queues monitored by each device driver process for new events, but the posting of this small message would be done by the micro-kernel, so that rogue processes cannot mess with it).
Simpler but less performant I/O APIs, like the stdio of libc, can be implemented on top of a multiplex AIO API, like that of liburing.