Hacker News Reader

Comment by hinkley | Hacker News Reader

And let me state again for the record that all of these promises being made by container systems sound an awful lot like the promises I was offered by 'real' operating systems in the early nineties.

I think the only real difference is that there has been a sea change in public opinion on this kind of aggressive isolation by default being worthwhile.

But a hypervisor publishing a bunch of services that talk to the world and each other? Things are beginning to look a bit more like microkernels as time goes on.

View on HN

Plus now, the app devs, now all turned devops because talent is scarse and demand is high, get to handle the complexity of concurrency, networking and reliability. This adds to deployment/testing/development setup, and when you change stack or job, the architecture slightly different and you need to learn it all over again.

Funnily enough, we couldn't make the microkernel concept works despite an homogeneous, integrated environment as a basis. But we are fine building frankenstein OS images, hacking multiple graph-shaped systems to make them play together, and call that a success.

Now don't get me wrong, I understand perfectly the benefits of it. But somehow, it feels like we arrived at the right result by the worst possible path, and at a price way higher than it should be.

What if the pure microkernel approach is flawed and hypervisors offer a sensible middle ground?

Most of the people that rail against pure microkernels have zero experience actually building stuff with them. Those that do have that experience have hard proof that systems built in this way are more reliable and easier to build than others. Hypervisors have their own place in the hierarchy, they do not compete effectively with microkernels on their own turf.

The typical problem with microkernels is poor backwards compatibility, mainly due to POSIX. If you add POSIX compatibility, you don't gain any isolation, performance or security benefits of microkernels, and changing your thinking and your programs to gain those benefits requires significant work when they're written against the POSIX API.

Building them that way from scratch is pretty much just as easy as writing programs in UNIX, modulo the lack of tooling that is mostly POSIX-based.

If you don't mind this tangent question...

Is there a microkernel API that has more acceptance nowadays? In other words, where should anyone wanting to work on real world / enterprise systems based on microkernel look?

I don't know if that's the idea, but Genode works over a few microkernels.

(original Genode dev here)

You nailed it: From day 2 or so we imagined to develop Genode as the future common API for the diverse range of microkernels and showcased the approach with all L4 kernels we could get our hands on. The vision is still to provide a replacement for POSIX in the microkernel world ;-)

Regarding POSIX, we don't think that tools and applications developed for this interface do not fit the microkernel world but on the traditional systems level POSIX (and its tools) is not sufficient to make the best of out of the microkernel advantages. For example, we run libcurl, GPG, libarchive, and coreutils in an orchestrated scenario to implement a robust and secure deployment/installation subsystem in the Genode-based Sculpt OS https://genode.org/documentation/articles/sculpt-19-07.

seL4: https://sel4.systems/

The kernel is even formally verified, and it's very widely deployed.

Define wildly.

The Okl4 kernel in iPhones is extremely limited in its functionality, you could probably run that code even without an Os. It could also be argued if in its current form it is really a microkernel. Furthermore, sel4 and okl4 are not the same product.

(nothing bad on sel4, I have used it myself and I personally know the people behind the project. But it's not what you think it is)

Not what I think it is? It's a capability secure microkernel, that's literally all I've ever claimed. I'm not sure what you think I've been saying.

seL4 looks quite interesting. What use cases have you used it in?

Isn't Windows NT the poster child for a non-POSIX mainstream microkernel?

NT kernel is not really a microkernel. It tool inspiration from microkernels but it's still a highly modular monolithic kernel.

Would you consider the gnu hurd folks experienced enough?

No. I thought GNU Hurd was a great idea executed poorly. QnX is that same idea executed in an excellent and pragmatic way.

But qnx is almost universally hated by people actually working with it :)

I've developed for it and I thought it was pretty nice. Sure, the kill command is called slay, and other oddities, but at its core it is a very well-designed system.

> Things are beginning to look a bit more like microkernels as time goes on.

Hypervisors typically are microkernels. The very first "hypervisor" was L4 in fact, with L4Linux.

Also, you're right that virtualization and containers are fulfilling the promises of operating systems. The problem of earlier OSes is that they didn't take security seriously enough. Modern hypervisors aren't much better at isolation than real microkernels like L4, KeyKOS and EROS. They're better than containers though.

> The very first "hypervisor" was L4 in fact, with L4Linux.

The term "hypervisor" was coined by Popek and Goldberg in 1974. L4 didn't appear until the 90s.

> Hypervisors typically are microkernels.

Except they have vastly different histories (look up IBM's VM) and uses and underlying technologies.

Here's a good post on the difference:

https://utcc.utoronto.ca/~cks/space/blog/tech/HypervisorVsMi...

> Microkernels are intended to create a minimal set of low-level operations that would be used to build an operating system. While it's popular to slap a monolithic kernel on top of your microkernel, this is not how microkernel based OSes are supposed to be; a real microkernel OS should have lots of separate pieces that used the microkernel services to work with each other. Using a microkernel as not much more than an overgrown MMU and task switching abstraction layer for someone's monolithic kernel is a cheap hack driven by the needs of academic research, not how they are supposed to be.

> (There have been a few real microkernel OSes, such as QNX; Tanenbaum's Minix is or was one as well.)

> By contrast, hypervisors virtualize and emulate hardware at various levels of abstraction. This involves providing some of the same things that microkernels do (eg memory isolation, scheduling), but people interact with hypervisors in very different ways than they interact with microkernels. Even with 'cooperative' hypervisors, where the guest OSes must be guest-aware and make explicit calls to the hypervisor, the guests are far more independent, self-contained, and isolated than they would be in a microkernel. With typical 'hardware emulating' hypervisors this is even more extremely so because much or all of the interaction with the hypervisor is indirect, done by manipulating emulated hardware and then having the hypervisor reverse engineer your manipulations. As a consequence, something like guest to guest communication delays are likely to be several orders of magnitude worse than IPC between processes in a microkernel.

> Except they have vastly different histories (look up IBM's VM) and uses and underlying technologies.

Microkernels over time have also employed vastly different technologies. All the "differences" you note are "intended system design/interaction", which frankly isn't meaningful. For instance:

Who cares how they interact with it? A microkernel is defined by the sorts of abstractions it provides and the isolation properties those abstractions entail. Hypervisors are effectively less expressive microkernels.

>"Using a microkernel as not much more than an overgrown MMU and task switching abstraction layer for someone's monolithic kernel is a cheap hack driven by the needs of academic research, not how they are supposed to be."

I found this interesting. Can you or anyone else say what the context was where academic researchers have needed to do this? What problem was it solving for them in a cheap way?

There is a linked article on his blog https://utcc.utoronto.ca/~cks/space/blog/tech/AcademicMicrok... that expands on this. Specifically:

>the whole 'normal kernel on microkernel' idea of porting an existing OS kernel to live on top of your microkernel gives you at least the hope of creating a usable environment on your microkernel with a minimum amount of work (ie, without implementing all of a POSIX+ layer and TCP/IP networking and so on). Plus some grad student can probably get a paper out of it, which is a double win.

This is a great read. Thank you.

Isn't the important thing about KeyKOS and EROS capabilities, and L4 variants aren't all capability-based, are they?

I believe it's because at some point the operating systems (particularly Linux based ones) became semi-closed ecosystems of software, instead of a platform to run other people's software. Also became a 'singleton' mentality too, where the only one version or variant of an app/library is accomodated.

So now we're falling back to the few stable ABIs we have available -- Linus's kernel ABI (containers) and some kind of x86 platform virtualization. I think it's a little sad that this happens because operating systems made a poor job of doing what we needed them to do.

This is simply a side effect of ill-disciplined programmers breaking ABIs whenever they feel like it. It's quite possible to maintain ABIs in the long term and upgrade and deprecate them thoughtfully, and if you do that then you don't need multiple versions of a library or service around.

This is totally true, but I think it's the opposite approach that would have actually made a difference here. Saying "we know these ABIs won't be stable, either practically, or by design, so this is where the OS ends"

Then Linux package managers would have made it easy to eg. install local (eg. per user, or in a folder) libraries or software.

Of course it was always possible, setting various $PATH variables etc. but because it wasn't easy (and you're immediately out of the ecosystem) all focus has been on the root user and singleton approach. Static linking was put to one side too.

In doing so we made the presentation of the "operating system" a full ecosystem of libraries, too complex to be a defined layer to interface with. But the kernel realised the importance of stable ABI layer, so now we have containers instead.

I install pip dependencies for python into a vendor folder. It is definitively the cleanest solution compared to dicking around with virtualenv

This is actually a great explanation of how/why we got virtualization rather than microkernels. The hardware presented an ABI that couldn't be circumvented. The problem with software interfaces is that they're soft. With h/w we use a 'good enough' interface for possibly too long before it gets replaced as where in software we decide to upgrade the as-yet-not-widely-adopted interface with a better one and repeat.

If the POSIX committee couldn't deliver a solution that was modern and backwards compatible, what hope do we have that developers will selforganize and ever do that?

POSIX committee doesn't deliver anything.

POSIX is just a common API across UNIX clones.

And besides the AT&T original design, UNIXes have disagreed in almost everything else, hence UNIX wars.

Q1. What is the Portable Application Standards Committee (PASC)?

The IEEE Computer Society's Portable Application Standards Committee (PASC) is the group that has and continues to develop the POSIX family of standards. Historically, the major work has been undertaken within Project 1003 (POSIX) with the best known standard being IEEE Std 1003.1 (also known as POSIX 1003.1, colloquially termed "dot 1"). The goal of the PASC standards has been to promote application portability at the source code level.

http://www.opengroup.org/austin/papers/posix_faq.html

Examples of APIs actually proposed by POSIX, that weren't already in widespread use before adoption by standard updates?

I think you're just making my point.

Not really, last serious revision was in 2008, or so.

Yeah all those stupid fools... who needs machine VMs, language VMs, processes, process namespaces... Just compile your software with pointer authentication and run it in kernelspace on bare metal. Why is everyone so stupid and I am the only smart person on this planet?

... If you haven't realized it yet. I'm not serious at all ...

Reply on news.ycombinator.com