Hacker News Reader

Comment by KMag | Hacker News Reader

The dirty secret is that we take a microkernel, rename the syscalls as "upcalls", throw in some hardware emulation code as a less efficient and uglier API for software that can't be bothered to be ported, and call it a hypervisor.

Also, your phone's baseband processor almost certainly runs a microkernel, and likely so does your car.

View on HN

Any recent iPhone/iPad, Apple Watch, the iMac Pro, and Macbook Pro touch bar, all contain Apple's "Secure Enclave", which runs the L4 microkernel:

> The Secure Enclave runs an Apple-customized version of the L4 microkernel. This microkernel is signed by Apple, verified as part of the iOS secure boot chain, and updated through a personalized software update process.

https://www.apple.com/business/docs/iOS_Security_Guide.pdf

I wonder which L4 they use. L4 is more a family than a single kernel. The different variants are essentially totally different kernels from the same school of thought.

For that security I'd assume seL4, though I did not see anyone obviously apple on the devel mailing list...

Looking into it more, it looks like a heavily modified fork of L4Ka::Pistachio. That's pretty fun, I may have to go exploit hunting this weekend.

https://www.blackhat.com/docs/us-16/materials/us-16-Mandt-De...

Same for many Android phones and AMD chips with PSP (trustonic's l4 forked OS).

And what many seem to be unaware, is that project Treble made Android Linux into a microkernel where drivers use Android IPC to talk with the kernel.

https://source.android.com/devices/architecture/kernel/modul...

https://source.android.com/devices/architecture/hidl/

So everyone running Android Oreo or newer on their phones, not only has a microkernel on their basebase radio, they also have a Linux tamed into a microkernel on their main ARM SOC.

I don't see a microkernel architecture here. Requiring drivers to be loadable modules is just arriving in the 1990s of monolithic kernels. Also, only new devices with SoCs introduced for Android 9 are required to use kernel modules and kernels newer than 3.18. [1]

This means most (updated) Android 8 devices in the field are not Project Treble and are running old-style module-less kernels and don't have A/B system partitions.

HALs only seem to be there to abstract over the APIs of different hardware implementations and access to them. Not necessarily their drivers.

I would love to be wrong about this, so I would really appreciate evidence to the contrary here.

[1] https://source.android.com/devices/architecture/kernel/modul...

Because you just stopped at the kernel modules part and haven't spent time reading how HAL in Android actually works, starting at the HIDL link.

There are two HALs in Android.

The old HAL, previous to Treble, which was hardly used by OEMs.

And the new HAL, which is enforced by Treble.

On the new HAL, drivers are implemented as Android services using Android IPC via interfaces defined in HIDL, or by using the new shared memory APIs introduced in Oreo as well.

https://source.android.com/devices/architecture/hidl/service...

https://source.android.com/devices/architecture/hidl/binder-...

https://source.android.com/devices/architecture/hidl/memoryb...

https://source.android.com/devices/architecture/configstore/

I actually looked into the HAL/HIDL docs before posting and found no evidence that they are there to implement drivers in userspace.

As I understand it HALs and HIDL are used to provide a standardized way to implement new device features in a compatible way. So a vendor introduces a odor-sensor and can define an HIDL interface for the userspace to call the device driver provided in a kernel module.

I do not see a requirement to implement drivers in userspace, nor provisions for it, like interrupt handling, i2c/spi access or similar.

HIDL is the basis of Android IPC between processes, known as Binder.

"Binderized HALs. HALs expressed in HAL interface definition language (HIDL). These HALs replace both conventional and legacy HALs used in earlier versions of Android. In a Binderized HAL, the Android framework and HALs communicate with each other using binder inter-process communication (IPC) calls. All devices launching with Android 8.0 or later must support binderized HALs only.

Passthrough HALs. A HIDL-wrapped conventional or legacy HAL. These HALs wrap existing HALs and can serve the HAL in binderized and same-process (passthrough) modes. Devices upgrading to Android 8.0 can use passthrough HALs. "

https://source.android.com/devices/architecture/hal-types

Here are the user space driver APIs for i2c/spi access on Android Things.

https://developer.android.com/reference/com/google/android/t...

Android Things is not included in normal Android as it seems. Android Treble/9 is just not even close to a microkernel.

It if was, there would be documentation on how to write a userspace device driver, but there isn't.

Binder allows for endpoints in kernel space.

Of course, otherwise how would they support passthrough legacy HALs for devices upgrading to Oreo as described on the documentation?

Interprocess RPC is only for Oreo and newer.

The current HAL works via dynamic libraries that get loaded into your process and talk to kernel space with a platform specific API. The passthrough support is just opening both sides in the same process, loading those legacy libraries, and wrapping the libraries in the treble API. So it actually uses IPC where both sides are in user space. That's what they mean when they describe it as 'in process' in the documentation.

Going forward, I expect the vendors to modify their kernel drivers to export the treble API over binder IPC directly while in kernel space.

So I wouldn't be surprised if it ends up that interprocess RPC only ebds up getting used in systems before Oreo.

I'll admit to not being steeped in the terminology, so feel free to educate me.. but some quick Googling suggests nobody else has called Treble a "microkernel." There are hits about Fuchsia (a totally separate OS), and some Android forks on microkernels.

I'm guessing that the Treble modules expose the traditional /dev and /sys interfaces like before, and these new HALs talk to devices through those, right? Is that not still a runtime-linked monolith, with fairly-thin userspace services between the kernel and the framework?

The bulk of driver code logic (e.g. interfacing with a touchscreen controller) is probably still running in the kernel - show me if I'm wrong. I would think this would be how you define a microkernel. One exception may be the GPU -- those drivers are often very thin in the kernel because of GPL, with a fat userspace library that they can legally keep closed-source.

They use Android IPC, which is a kind of RPC between processes.

https://source.android.com/devices/architecture/hal-types

This is exactly how I understand it.

Recently committed Gasket (Google ASIC Software, Kernel Extensions, and Tools) kernel framework is going even further in that direction.

https://lwn.net/Articles/758745/

> Could allow for APK installable kernel drivers, allowing modules to be upgraded (even live)

https://twitter.com/MishaalRahman/status/1029064974805688320

Interesting thanks for sharing. Looking forward for when it pops up on AOSP.

If it pops up on AOSP.

So, MkLinux 2?

and according to someone, your Nintendo console too

It's quite crazy to count all the layers

    cpu microcode
    cpu isa
    hypervisor
    os kernel
    userspace api
    browser runtime
    js vm
    application
    your mouse

I want my 386 back

If you mean the Switch, yes it is a microkernel, so fast enough to play all those flashy Vulkan/NVN games.

https://media.ccc.de/v/34c3-8941-console_security_-_switch

Soon it will be:

    ...
    webassembly vm
    (insert variable height custom stack)
    application
    ...

Sad times.

Oh I forgot about wasm. Maybe one day someone will just collapse the whole thing and embed the internet in silicon.

"This, Jen, is the internet."

https://youtu.be/iDbyYGrswtg

https://utcc.utoronto.ca/~cks/space/blog/tech/HypervisorVsMi...

> Microkernels are intended to create a minimal set of low-level operations that would be used to build an operating system. While it's popular to slap a monolithic kernel on top of your microkernel, this is not how microkernel based OSes are supposed to be; a real microkernel OS should have lots of separate pieces that used the microkernel services to work with each other. Using a microkernel as not much more than an overgrown MMU and task switching abstraction layer for someone's monolithic kernel is a cheap hack driven by the needs of academic research, not how they are supposed to be.

> By contrast, hypervisors virtualize and emulate hardware at various levels of abstraction. This involves providing some of the same things that microkernels do (eg memory isolation, scheduling), but people interact with hypervisors in very different ways than they interact with microkernels. Even with 'cooperative' hypervisors, where the guest OSes must be guest-aware and make explicit calls to the hypervisor, the guests are far more independent, self-contained, and isolated than they would be in a microkernel. With typical 'hardware emulating' hypervisors this is even more extremely so because much or all of the interaction with the hypervisor is indirect, done by manipulating emulated hardware and then having the hypervisor reverse engineer your manipulations. As a consequence, something like guest to guest communication delays are likely to be several orders of magnitude worse than IPC between processes in a microkernel.

People never seem to want to admit that sometimes, technology just dies, or at least becomes obscure, and so will invent bizarre "connections" between what we used to be working on and what we have now.

OKL4 was both a microkernel and hypervisor. They coined term microhypervisor to differentiate that.

https://microkerneldude.wordpress.com/2008/04/03/microkernel...

NOVA is an open-source micro-hypervisor available in GenodeOS that similarly combines hypervisor functionality with a microkernel-like design:

http://hypervisor.org

The Xen hypervisor also got many of its design elements from the Nemesis OS, which had a lightweight kernel. There's plenty of overlap possible. A number of research and production systems are also both hypervisors and microkernel-like, too. So, it's more an existing concept than a bizarre, hypothetical connection. For OKL4, also deployed in over a billion phones they claim.

The fact people disagreed by downvoting rather negates your points.

Your second paragraph is exactly what I meant by adding hardware emulation code to create a less efficient and uglier API.

The hardware-like interface to the hypervisor is both inefficient and not very abstract. For instance, a guest kernel attempts to perform a series of manipulations of the page table, each of which traps to the hypervisor, or else it performs an equivalent upcall to manipulate the page tables.

The hardware emulation code ads a lot of potential bugs to the hypervisor. I hope our current hypervisors are an evolutionary transitional form and in the future we'll interact with hypervisors solely through upcalls.

Reply on news.ycombinator.com