Why unikernels might kill containers in five years

Id point you to osv.io havent really played arounf with iy much though the concept is pretty intriguing .Being able to run the kernel directly on the hypervisor , and containing only your app and whats needed to run , thus obviatibg the need for device drivers with everything running in the kernel address space.Also check out LING for the erlangVM

The idea is that your security boundary is at the VM. The VM hypervisor presents a virtual CPU; malicious code cannot jump out of the box.

Many common server frameworks build in multiprocessing to the language runtime: Node.js mandates an event-driven callback style, Go has goroutines, etc. So it already exists in userland. Unikernels avoid duplicating that functionality at the kernel level, so the two systems don't fight each other.

The hypervisor proxies device drivers through from the host operating system (running in Dom0) to the guest operating systems.

The idea is to run eg. Node, or Postgres, or memcached on bare metal. So instead of write() trapping into the operating system, it becomes a library function that directly talks to the underlying device driver. The only process the VM runs would be that one application, and you'd rely on the hypervisor to schedule multiple applications on one physical box.

But if the hypervisor already has process separation, quota management, device drivers, etc. there's no point duplicating it all in the VM.

The advantage comes from unikernels being even more disposable than containers. Your kernel is built with an absolutely tiny number of options. You wouldn't even need things like the console driver, because if a unikernel crashes, you just restart it, with crash logs sent somewhere for analysis. You can remove a lot of your networking stack, because your HTTP server, that only ever speaks http, can get rid of base things like UDP support. You don't need a full-featured filesystem, because things are only written to at compile time. The number of syscalls you expose can equally be tiny. In short, you have an absolutely minuscule kernel.

From there, you can start stripping out other things. You don't need a huge init system because all you're doing is running a single application. Your libc can be made tiny, or removed completely, because you can strip it of all the calls your app doesn't need. You don't need to run an sshd, because there is no shell to get into. You can get rid of PAM because there's no accounts to access or authenticate against.

On real machines and VMs, these things are necessary so you can run tests, do cleanup, etc. Unikernels are centered entirely around the idea that those things are someone else's job, and if needed, can be shouted for across the network.

Unikernel frameworks like Mirage either provide all those features, or in the case of drivers, are just targeting virtio/veth/etc. paravirtualized devices offered by the hypervisor. So the domU doesn't have any abstraction to worry about other than just targeting a particular Hypervisor's representation of storage and networking.

Xen provides very basic (but safe) message/event channels between VMs. Basically this becomes your IPC, and the VMs become user processes.

With a new enough CPU with the correct memory virtualization features, xen can scale to many thousands of domUs. And with each domU only taking up slightly more resources than a typical user process, this is a completely reasonable approach.

Again, your Hypervisor becomes the OS which has to worry about the baremetal, and the VMs replace user processes.

I intend to playing with MirageOS seriously myself next year.

Edit: some dependencies in an example MirageOS app: https://mirage.io/wiki/technical-background#ModularOSLibrari...

> They can't really get away without device drivers

I agree with you 100%, but it does make me think. Every Linux installation I've done in the last five or so years has been either under ESXi, or on AWS.

I'm sure I'm not alone in that, and that's a trend that's going to continue to grow.

How much of the kernel is drivers that are absolutely never going to useful in that scenario?

How much room is there for distributions to start supplying a kernel configuration with half of it never built?

You joke but one nice use of them is a simplified system interface which can allow a traditional OS to give such processes significantly lower privileges, e.g. to sandbox something like mplayer. Unikernels make it possible to play with where you put a bunch of code that would otherwise need to be in the kernel or reimplemented.

Sure. With the true AI around the horizon if you wait long enough you won't have to figure anything - it will do it for you.

That's the case with most technologies.

I managed to skip the whole Angular/Mongo hype, and now that I'm actually getting back into frontend development, it's all about React. Can't say I missed it.

I remember telling my boss at my very first job "Yeah, I want to learn MFC and COM". His response was "Why? By the time you're out of college, everything will be .NET." It turned out he had underestimated: by the time I graduated from college, MFC & COM had been replaced by .NET, which had been replaced by webapps running under J2EE, which were just about to be replaced by Rails & Django.

As far as security is concerned a unikernel with lower attack surface will beat out a general kernel. A unikernel for AppX will have a completely different attack surface from AppY so most generalized attacks just won't work. Shell attacks like heartbleed won't work because there is no shell. Given that there's a single process, even if you did compromise the system there's not a whole lot you can do with it.

Apparently unikernel images can be as small as 200KB.

http://blog.acolyer.org/2015/01/13/unikernels-library-operat...

That might be only for niche cases, though, but others seem to say it can be under 5MB for more general uses, too:

http://www.techrepublic.com/article/unikernels-offer-a-strip...

This is a nice talk on unikernels:

https://www.youtube.com/watch?v=oHcHTFleNtg

A unikernel can be pretty small, the sample one that comes with Xen builds to 230KB.

Some of the OpenMirage builds are around 25MB.

Regarding security, any distro install is going to have an attack surface several orders of magnitude higher than a unikernel. Does anyone know how the quantity of exploitable vulnerabilities varies with attack surface (ie. SLOC)?

The original poster says:

"Given that unikernels compile only which is necessary into the applications, the surface area is very small."

I think this is a somewhat strange statement. You don't lower the surface area by not compiling in dead code--unless you're offering a way to run arbitrary code, in which case it can help implementing the anti-features the attacker wants. But to me it seems the primary innovation of unikernels is that they are not implemented in C, but in a higher-level language like Ocaml, Erlang, etc. Those languages, if their compilers and runtimes are implemented correctly, don't allow buffer overflows, and have also other features to prevent arbitrary code execution (type system, or secure process isolation). Those kernels have (given the same attention to quality) the potential of being more secure than the Linux kernel by using such languages. Also, by not having to be a general OS kernel, they are simpler to implement, hence reducing the number of bugs likely to be around.

It's not the "compiling into the application", it's the "implementation of the algorithms" that's smaller. Perhaps that's what the original poster meant, but it sounded strange to me.

At Boxfuse (https://boxfuse.com) we decided to combine the best of both worlds: dynamically generated minimal Linux-based images. They are just a few MBs in size, generated in seconds and start almost instantly, yet retain the tried and true quality and compatibility of the Linux kernel.

We coupled that with a secure repository and blue/green AWS deployments for a great out of the box experience.

Unikernels are essentially the far less interesting cousin of exokernels, in that they nominally preserve the libOS angle but are limited to application deployment under a bare-metal hypervisor.

Much of the motivation for microkernel research, too, was for applications to be able to control their workloads at a finer granularity while retaining a full OS environment. Scout and SPIN were such examples. Something as simple as making page replacement and eviction a userland server where each library or application can implement its own policy over the RPC interface, can yield great benefits.

No, just realising the ones we have based on assumptions from the 70s don't match what we really want.

My phone has an operating system that mostly pretends that it's a mainframe. With just a little hardware, I could hook some circa 1978 VT100s up to it and a team of average *nix web backend developers could even be productive.

My "workstation" pretends to be several tens of machines doing all sorts of different things. These boundaries exist for no reason other than to map to organizational concerns -- my CPU does all kinds of computational gymnastics so I can "keep things tidy".

The model is just wrong. Things are immutable that shouldn't be. Things are mutable that shouldn't be. Things aren't distributed that should be. Things aren't durable that should be. Things are shared that shouldn't be. Things are isolated that shouldn't be. Config management, containers, orchestration, stuff like adding BPF to the kernel, Docker, Hadoop, Mesos -- all very cool, but all very much bandaids.

We're working on it but because we're human we can't go straight for the goalpost, we have to play politics, cheat by modeling the future using what we already have today and lead by example.

> Are we essentially witnessing the death of the operating system?

It depends what you mean by "death". Chester's First Law applies. Some code, somewhere, is responsible for the services traditionally provided by an OS.

Definitely not. Where do the hypervisor calls of your unikernel end up? In the OS that is running in the Dom0.

If anything this is not the death of the OS, it is the death of POSIX as an application API.

Would depend on the choice of unikernel and how theyr doing their stuff osv.io for example basically run the app in kernel space on top of a hypervisor and inherently as hardware agnostic as possible.

Container images don't have a kernel, but they do have userland portions of an OS (e.g. libc).

AFAIU the whole thing runs in ring 0.

Mirage can be configured to write trace data to a ring buffer shared with another VM, where you can run a visualisation tool to explore it. For an example, see:

https://mirage.io/wiki/profiling

Of course, having such a small (and legacy-free) OS makes debugging vastly easier in the first place.