NVMM: A full, fast and flexible virtualization stack for NetBSD | Hacker News Reader

NVMM: A full, fast and flexible virtualization stack for NetBSD

blog.netbsd.org

150 pointspmarin24 comments

I wonder why each of the three major BSDs ended up with their own hypervisor implementation instead of sharing their code. There is now bhyve on FreeBSD, vmm on OpenBSD and now also NVMM on NetBSD. Is this kind of diversity a good thing? Interestingly, the non-BSD illumos has imported FreeBSD's bhyve (and their bootloader). I assumed that these smaller OS-communities would be more inclined to share the work among each other but for some reason this did not happen with their home-grown hypervisors.

NetBSD, OpenBSD and FreeBSD are about as equal as French, Italian and Spanish are. All of them are derived from Latin (BSD), but they have diverged so much over time that they are effectively their own things.

What happened to PopularBSD?

Different goals and the internals of each BSD have diverged pretty heavily.

Yes. The goal of OpenBSD vmm is primarily simplicity and a small, easily audit-able codebase. FreeBSD’s solution would not meet OpenBSD’s requirements and vice versa.

I understand the different goals part, but it would be nice if these technologies could run on other BSDs (and Linux). bhyve and vmm are exclusive to FreeBSD and OpenBSD respectively AFAIK.

From a general generic user perspective, you're right that that would be useful. As a user, I also understand that frustration.

Basically, it comes down to having very little money, if any, to fund these tasks and most of the work is done by volunteers. It is much easier and faster to work with the local environment than manage kernel portability issues between different BSDs and even Linux.

The cool thing is that anyone is free to help with the project and port things if they choose. It is truly an issue of funding and finding folks willing to do the work (and maintain it).

It is a lot of work and specialized knowledge across different kernel domains.

Is it wrong to have different virtualization solutions out there though? It's generally nice to know that people are trying to enhance how virtualization works on multiple platforms to suit their particular itch and we can all benefit from these different ideas being tried so we can choose the best pieces of them over time. Objectively, we want these virtualization solutions to be suited to the kernel/pipeline they are running on to give the expected behaviour and performance for the guest workload.

From the end user side things like libvirt have done a lot to make the interface to these systems consistent, and so I really don't feel a ton of pain moving between virtualization solutions anymore as an end user compared to having to deal with xen/qemu/vmware/virtualbox/hyper-v incantations. If I'm just trying to run a workload I don't care about NVMM, unless it has a particular feature that I need for network emulation like say RapidIO, Fibre Channel or Infiniband as an emulated link layer.

> bhyve and vmm are exclusive to FreeBSD and OpenBSD respectively AFAIK.

What you are saying may be true for OpenBSD/vmm, but Bhyve has been ported to Illumos, and by extension, SmartOS & OmniOS. Apple MacOS has Xhyve, which was inspired by and shares some code with FreeBSD Bhyve.

The second part is the plumbing is different these days. The split has gone on for a long time.

If you take this a step further then you could as well ask why there are at least four BSD forks at all.

Or, equally, why there are multiple Linux distros.

Why not implement the actual Linux KVM ioctls instead of a something similar yet different? You could then run qemu on top and get an actual "full stack" (ie. everything up to Open Stack or Kubevirt).

They tried this with Illumos (the OpenSolaris descendant which itself is a distant BSD descendant), but found that maintaining their fork of QEMU/KVM code was more difficult than porting over Bhyve from FreeBSD.

(Open)Solaris is a SVR4 descendant; not a BSD descendant. SunOS however (until version 4, version 5 being Solaris) is a BSD descendant.

Thank you for the details. Now we all know more.

Sounds a bit like the hypervisor framework on MacOS. They’re is xhyve (a port of bhyve) for Mac on that.

Probably could port bhyve to this too? Haven’t looked.

The article says as much: "It has some similarities with WHPX on Windows and HVF on MacOS."

So, supposedly, NVMM could be to a NetBSD bhyve port what HVF is to the xhyve on MacOS?

Maxime Villard gives us another awesome gift :) Thanks!

Is this similar in purpose/function to bhyve?

From the article:

When it comes to the userland emulators, NVMM does not provide one. In other words, it does not re-implement a Qemu, a VirtualBox, a Bhyve (FreeBSD) or a VMD (OpenBSD). Rather, it provides a virtualization API via the libnvmm library, which allows to effortlessly add NVMM support in already existing emulators.

Yes.

Next step, it'd probably be userful add support for it to Libvirt.

That way the large amount of cross-platform virt software around - much of which is written to use Libvirt - could interface directly with it. :)

Reply on news.ycombinator.com