Unikernels
github.com
github.com
That said, there are options for running unikernels as K8s workloads if you want, eg NanoVMs: https://docs.ops.city/ops/k8s
There is definitely some performance overhead, but in most cases it is less than hypervisor-based approaches.
https://gvisor.dev/docs/architecture_guide/performance/#syst...
IO bound tasks can be up to 10x slower using ptrace. I think using hardware acceleration gives you acceptable performance but ptrace is just a non-starter for prod.
If the hypervisor is KVM, which they are if running on modern AWS EC2 instances or GCP, unikernel apps are literally just Linux processes; the underlying Linux host is doing all the heavy lifting. Conceptually, they're essentially the same as a sandboxed ordinary Linux process with an in-process IO stack, but without the ability to monitor or debug them as if they were an ordinary Linux process.
GDB works great as does printf and others: https://docs.ops.city/ops/debugging
Prometheus and other open source monitoring solutions work out of the box and we even have a custom APM service that is unikernel-tuned https://nanovms.com/radar .
App --"syscall"--> gVisor --syscall--> kernel
Unikernel --hypercall--> kernel
Though ideally something like SR-IOV would come into play and they hypervisor is just scheduling shared compute. Of course there is theory and reality and reality is unikernels never really caught on for many reasons while the normal stack just got optimized enough.
It is a matter of who is offering what, not what unikernels are capable of.
Basically when you activate Hyper-V, you will be getting one VM running where the host is only a guest with special privileges known as root partition.
> If they are running on Hyper-V on Azure, there is no underlying kernel doing anything.
In a Xen model as I understand it, the dom0 kernel is still actually responsible for talking to all the hardware directly and presenting a virtualized implementation that Xen can mux other guests on, no? So there’s still a kernel there and it’s doing quite a bit of work, no?
[0] - https://docs.microsoft.com/en-us/windows-server/virtualizati...
Some devices they can also support hardware assisted virtualization like for PCIe devices (NICs/NVMe storage/GPUs) via SR-IOV but it's been pretty rare to see that in practice with unikernels as they typically have limited physical device driver support on top of that not really being an option everywhere all the time as it places limitations on the cloud provider that paravirtual devices don't.
https://docs.microsoft.com/en-us/virtualization/hyper-v-on-w...
I think you might be right. Right place at the right time for containers.
It turns out that basically nobody cares about either of those. I know someone who for a while worked at a company that was trying to make money off unikernels. They ended up re-implementing a database in a unikernel, in a way that gave them a 2.5x performance improvement over the original project -- or to put it a different way, switching should allow companies to cut their hosting costs in half. Even with such a clear "win", it was still a difficult sell.
Think about how much backend code is written in interpreted languages like Python, or PHP, or Javascript, rather than in compiled languages like Go or Rust. It's just simpler to start with the simple solution and then throw money at the problem as you scale. And while performance may be one of the reasons that people are choosing Go or Rust for backends, if it were the only advantage, it's unlikely that would be compelling enough.
And its a shame AWS doesn't allow volumes with sizes smaller than 1 GiB that i understand you can get really small images with Unikernels
The 1 gig limitation on clouds is not such a huge deal though as we can upload your image, however small it is as that size and then tell the cloud to provision the disk the size you need which acts kind of like a sparse file (but technically not one).
Since recompilation is necessary anyway for unikernels, syscalls could be replaced by function calls or some other user mode thing instead of trapped. It would allow entire containers to run as processes. Not that interesting for cloud, but very interesting for distribution to endpoints or self-hostable apps.
That's far from enough. It's conceptually and practically wrong to rely on the application to monitor itself.
> IDS makes no sense when you don't have logins and such
On the contrary, there is plenty that IDS can do for webapps.
As for the IDS question/statement - can you explain in more detail? Are you talking about file integrity checks or? Unikernels don't have the concept of users or shells or remote login or many of the things that an IDS would actually be looking at.
If it was something such as an attacker overwriting a shared library and you want to monitor or ensure that can't happen both of those operations are feasible in unikernels.
Also a lot of hardware and VM management software that perform remote administration functions, e.g. asset tracking, reacting to low batteries on UPSes, monitoring network health...
It's absurd to think that a whole OS worth of code should be jammed into the application or the unikernel. That's what traditional kernels are for.
citation needed
> And the tools are rarely installed on "cattle" anyway
citation needed
At the end of the day your application decides how much memory it wants and the sysadmin/SRE/devops person just ensures it has enough so it doesn't crash.
If you are hosting your own workloads and those workloads only need tens of megs of ram than you can pack as much as your hypervisor can handle.
Alfred talks about booting 110,000 vms on one host before memory exhaustion:
Operating systems already solve this problem relatively well, without the overhead, via processes and containers.
Instead of AWS (hypervisor) => linux => k8s => containers unikernels advocate for AWS (hypervisor) => unikernel and that makes them run much faster in general (we've clocked upwards of 300% req/sec for go/rust webservers on AWS for instance) and a lot safer.
You're misunderstanding the levels of abstraction probably because the word "kernel" within "unikernel" is throwing you off. The idea is to use a partial kernel (only the minimum services one needs). The so-called "kernel" is a library of code where you compile the minimum bits into the single-process image.
>Operating systems already solve this problem relatively well, without the overhead, via processes
A full operating system like Linux is expending extra overhead to schedule/prioritize/monitor processes (plural) -- because Linux is designed to be more general purpose and open-ended than a specialized unikernel. In contrast, a unikernel with only 1 singular process (say a specialized db engine) doesn't need to expend extra cpu on processe(s) scheduling.
All that said, it doesn't seem like unikernels have enough advantages to attract widespread adoption like containers.
I think the value isn't in the containerization vs unikernel comparison. If you're using containerization you've accepted certain security risks. Where unikernels have a lot of potential IMO is in high security environments where the security risks of containerization are not acceptable.
Coming from k8s/firecracker it is common to think that you need to orchestrate your unikernels with a framework of some kind. In our case (Nanos/OPS) a lot of people think that means spinning up an ec2 linux, sshing in and using 'ops run' on top of that but that is never suggested for prod deploys. Instead we suggest doing an 'image create' followed by an 'instance create'.
What does this mean? Essentially every time you hit the deploy button a new ami is made and a brand new ec2 instance spins up without any linux inside. So instead of adding layers through containers we actually subtract them. That means you can still configure the instance to your hearts content but you don't have to manage it - the cloud does for you and this is a huge win for many teams that don't want to deal with all the ops/SRE work that something like k8s brings (or even normal vanilla linux does).
It is important to realize that containers extract heavy performance penalties when running on top of existing infrastructure (like the cloud) since they duplicate storage and networking layers. They also have severe security issues - the shared kernel being the main one.
What if your application involves multiple processes and threads?
How many more comments will we read yet another copy of a nanos sales pitch?
The post is about unikernels, so, obviously every single comment will say something about them.
https://www.usenix.org/legacy/events/hotos05/final_papers/fu...
Darwin was never intended to be anything except what it is, which is a monolithic kernel. The XNU kernel was based on FreeBSD kernel and Mach kernels. Some versions of the Mach kernel were microkernel, but many were not.
Both NT and XNU incorporate message passing features from microkernels, but they are monolithic in that they are essentially a single large process.
"Hybrid kernel" is more of a marketing thing than an engineering term.
Microkernels are a dead-end and never stopped being a dead end. It's a lovely idea that didn't work out. They had limited commercial success in embedded systems, but only because those embedded systems didn't actually do very much and what they did was largely not performance critical.
And Apple's long term roadmap to move all kexts to userspace is a means to improve the current state.
VMware's vmkernel? VFIO/Kernel-bypass? Shoving things into kernel space does not guarantee good performance by any means, and it murders security.