VirtualBox KVM Public Release | Hacker News Reader

VirtualBox KVM Public Release

cyberus-technology.de

520 pointsCyberusTech196 comments

For the past few months we have been working hard to provide a fast, reliable and secure KVM backend for VirtualBox. VirtualBox is a multi-platform Virtual Machine Monitor (VMM) with a great feature set, support for a wide variety of guest operating systems, and a consistent user interface across different host operating systems.

Cyberus Technology’s KVM backend allows VirtualBox to run virtual machines utilizing the Linux KVM hypervisor instead of the custom kernel module used by standard VirtualBox. Today we are announcing the open-source release of our KVM backend for Virtualbox.

Open articleView on HN

Anyone using VitualBox please be careful about the extension pack. Oracle is very litigous.

https://www.reddit.com/r/sysadmin/comments/147k6az/oracle_is... https://www.reddit.com/r/sysadmin/comments/d1ttzp/oracle_is_... https://www.theregister.com/2019/10/04/oracle_virtualbox_mer...

We banned virtualbox in our organization since vmware workstation (or virt-manager) is way cheaper than dealing with oracle.

Exactly what I came to comment. Same thing here, they seemed to hyperfixate on the extension pack which most VBox users would need. We had to get everyone off that as soon as possible.

Of course this isn't limited to VBox, but their database as well. Just avoid.

Just noting that USB 2.0/3.0 support no longer requires the extension pack, and the list of features [1] that require the extension pack has been gradually decreasing. It may not be as necessary as it once was.

[1]: https://www.virtualbox.org/manual/ch01.html#intro-installing

USB was the only reason I ever had it. I need nothing from the current list.

What if you just ignored Oracle? Do they even have teeth? If they made a download free it should be free.

You can't hand someone a banana on the street and then come back 3 months later demanding $1000 for it.

If your organization uses any Oracle software then I'm certain that the organization has agreed to let Oracle audit it for license compliance at any time.

Oracle is that awful practices company. Their software audits are a notorious issue, and a good reason to never ever use software from them.

See the Oracle org chart for reference: https://www.unix.com/what-is-on-your-mind-/240815-org-charts...

Finally!

Every time I need to run a virtual machine, I choose libvirt because it's more performant and easy to deal with than Virtualbox (no kernel module, etc.), but the GUI choices are pretty terrible. The "best" libvirt GUI is virt-manager and it's very, very buggy and lacking features (i.e. doesn't play nice with HiDPI screens, no way of configuring IPv6, etc.)

Many times I have caved and chosen VirtualBox simply because at least it feels nice to use, even if not as performant as libvirt/kvm. Not anymore!

Great! What guests do you typically run where you see better performance with libvirt/kvm?

Mostly Linux, but also Windows when I had a VFIO passthrough setup. I don't think it's even possible to set it up with Virtualbox to have decent enough performance.

You can also setup VFIO in VirtualBox/KVM. We haven't polished it yet though. You can check the video here to see GPU virtualization in action:

https://www.cyberus-technology.de/products/hypervisor (Don't mind the English, we are not native speakers. :)

I'm really curious about this, is it gpu para-virtualization or actual VFIO requiring built in support from the gpu hardware?

We have used this with recent Intel GPUs that support SR-IOV. This is what you see in the video on the cyberus website. Intel hasn't managed to upstream the drivers for this yet and you have to piece together things, which is very unpleasant. But we are there to help, if someone wants to use this in a professional setting.

We used to have support for Intel GVT-g GPU virtualization as well, which was more of a software solution. This doesn't work with modern Intel GPUs anymore.

Thank you for the info. I use AMD consumer GPUs, none of them unfortunately support SR-IOV afaik, there are some developments from google around virtio-gpu(DRM native context) I've been following, I was hoping this was something similar.

I wonder if it would be possible to use DXVK here for windows guests to play those unruly games. That is have a pass through direct x driver in windows that sends all the commands to dxvk which either sends the image back to the vm or renders directly on the screen.

I thought virt manager was ok but honestly your complaints about it are specific and fair.

Virtual box has graphical configuration for a ton of different options. It also “just works” in many cases and is relatively easy to use.

I am surprised the open source community has not built better gui tools, and no project, closed or open has made configuring pcie passthrough easy.

I have always wanted to be able to run Windows in a virtualized session with my GPU for gaming, and use my onboard APU for the Linux host, but the configuration is daunting, and many of the games I play today don’t work on linux thanks to anticheat or DRM.

I wish there was a port of UTM to linux

Gnome Boxes is an attempt at a similar interface, but yeah, it's not quite as polished.

virt-manager supports more complex libvirt XML configurations, can also manage VMs created by Gnome Boxes, but doesn't yet have IOMMU/PCIE passthrough with OVMF UEFI device selector and vm configuration gui: https://virt-manager.org/

> I am surprised the open source community has not built better gui tools

This is the thing that isn't surprising.

In order to make a better GUI tool, you have to understand how the internals work. Then you don't care to use a GUI tool because you know how to use the command line or edit the configuration files. The people who want better GUI tools are the people who don't know how to make them.

In order to change this, the people who want GUI tools but don't know how to make them have to provide some incentive (typically money) to the people who know how to make them but don't want them, e.g. via donations or some kind of commercial agreement. And if they stand around and wonder why nobody else has fixed their problem for them instead of doing the thing that causes it to be fixed, an object at rest tends to remain at rest.

> no project, closed or open has made configuring pcie passthrough easy

"GPU passthrough with libvirt qemu kvm" https://wiki.gentoo.org/wiki/GPU_passthrough_with_libvirt_qe...

"PCI passthrough via OVMF" https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF :

> The Open Virtual Machine Firmware (OVMF) is a project to enable UEFI support for virtual machines. Starting with Linux 3.9 and recent versions of QEMU, it is now possible to passthrough a graphics card, offering the virtual machine native graphics performance which is useful for graphic-intensive tasks

KVM-GPU-Passthrough: https://github.com/BigAnteater/KVM-GPU-Passthrough

https://clayfreeman.github.io/gpu-passthrough/

I don't think that linking two different Wiki's (for different Linux distros) and two different github posts is "easy" compared to VirtualBox's very "fisher price" Next-Next-Next-Done GUI

Not saying I prefer one or the other, but it's worth bearing in mind where "the bar" is

FWICS from scanning those resources, there are a few shell commands to wrap with a config parser and an output parser for a GUI

E.g. virt-manager is built with glade XML and Python:

virt-manager/virt-manager//ui/createvm.ui: https://github.com/virt-manager/virt-manager/blob/main/ui/cr...

virt-manager/virt-manager//ui/gfxdetails.ui: https://github.com/virt-manager/virt-manager/blob/main/ui/gf...

virt-manager/virt-manager//ui/hoststorage.ui: https://github.com/virt-manager/virt-manager/blob/main/ui/ho...

virtManager/createvm.py: https://github.com/virt-manager/virt-manager/blob/main/virtM...

virtManager/device/addstorage.py: https://github.com/virt-manager/virt-manager/blob/main/virtM...

virtManager/device/gfxdetails.py: https://github.com/virt-manager/virt-manager/blob/main/virtM...

virtManager/addhardware.py:

  DeviceController.TYPE_PCI

    def populate_controller_model_combo(combo, controller_type):

https://github.com/virt-manager/virt-manager/blob/135cf17072... https://github.com/virt-manager/virt-manager/blob/135cf17072...

"Locating the GPU": https://clayfreeman.github.io/gpu-passthrough/#locating-the-... :

  for d in /sys/kernel/iommu_groups/*/devices/*; do
    n=${d#*/iommu_groups/*}; n=${n%%/*}
    printf 'IOMMU Group %s ' "$n"
    lspci -nns "${d##*/}"
  done;

iommu.sh gist: https://gist.github.com/Roliga/d81418b0a55ca7682227d57af2778...

iommu_groups.sh: https://github.com/drewmullen/pci-passthrough-ryzen/blob/mas... :

  lspci -nns "${d##*/}"

"PCI passthrough via OVMF > 2. Setting up IOMMU > 2.2 Ensuring that the groups are valid": https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#En... :

> An IOMMU group is the smallest set of physical devices that can be passed to a virtual machine. For instance, in the example above, both the GPU in 06:00.0 and its audio controller in 6:00.1 belong to IOMMU group 13 and can only be passed together. The frontal USB controller, however, has its own group (group 2) which is separate from both the USB expansion controller (group 10) and the rear USB controller (group 4), meaning that any of them could be passed to a virtual machine without affecting the others.

"Exporting your ROM": https://github.com/BigAnteater/KVM-GPU-Passthrough?tab=readm... :

  lspci -vnn
  find /sys/devices -name rom

  # PATH_TO_ROM=

  echo 1 > $PATH_TO_ROM
  mkdir -p /var/lib/libvirt/vbios/
  cat $PATH_TO_ROM > 
  /var/lib/libvirt/vbios/gpu.rom
  echo 0 > $PATH_TO_ROM

"Attaching the GPU" [with `virsh`] https://clayfreeman.github.io/gpu-passthrough/#attaching-the... :

  <hostdev mode='subsystem' type='pci' managed='yes'>
    <rom file='/path/to/gpu-dump.rom'/>
    <source>
      <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </source>
  </hostdev>
  <hostdev mode='subsystem' type='pci' managed='yes'>
    <source>
      <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
    </source>
  </hostdev>

"Adding your GPU and USB devices to the VM" [with `virt-manager`]: https://github.com/BigAnteater/KVM-GPU-Passthrough?tab=readm...

> 1. Add every PCI device which has to do with your graphics card to the VM.

That's still MASSIVELY more complex than "Next -> Next -> Next -> Done"

virt-manager plays fine with hidpi on Wayland. On the opposite side, VirtualBox’s GUI is super buggy in Wayland.

It’s basically the opposite for both. I use virt-manager because the GUI is simpler (and setting up virtualbox is a nightmare anyway).

Regardless, this feature is a step in the right direction. I’m wondering if distributions will pick it up or if it will ever be integrated upstream.

That's a very recent change to virt-manager, so recent it's on nixos-unstable, but not on nixos-23.11. So it might be in Fedora 39 and some rolling-release distros... and nowhere else.

Could you link the change? At least a few days ago virt-manager still seemed to have scaling issues with guest displays, on nixos-unstable. I had viewer scaling on though as a workaround, so maybe I just didn’t notice.

It's not perfect. On older versions, it'd sometimes add a black border the size of your scaling factor. It still reports your window size times your scaling factor as internal resolution with guest additions though.

100% agree about the terrible GUI choices. That actually turned me away completely from libvirt for virtual machines because when I actually needed to work with virtual machines I didn't want to finick around with CLIs and I stuck to VMware until recently. (at the time I ran Windows 10 on my main PC and Windows has a pretty terrible CLI)

Have you tried Gnome Boxes?

I have, it is very limited compared to Virtual Box.

Regarding IPv6, there is support. In fact I run IPv6 only networks including for the hypervisors.

    $ virsh net-dumpxml default6
    <network>
      <name>default6</name>
      <uuid>73590ea2-eb15-4e67-b104-319721bdf302</uuid>
      <forward mode='route'/>
      <bridge name='virbr1' stp='on' delay='0'/>
      <mac address='52:54:00:ff:a7:2d'/>
      <domain name='default6'/>
      <ip family='ipv6' address='2001:db8:ffff::1' prefix='48'>
      </ip>
    </network>

One can also use DHCPv6 if required.

In virt-manager?

Virt-Manager does allow you to edit the xml config file by hand from within the interface. So, yes you can use ipv6; but no, it's not a convenient point and click gui interaction.

I'd be curious to hear the specific reasoning behind it.

That's what I meant, I had to use virsh to set that up because it's not possible to do in the GUI.

But it is!

virt-manager v4.0.0 (Ubuntu 22.04 amd64)

"QEMU/KVM" > "Details" (via double-click or context menu) > "Virtual Networks" tab > "+" (Add) button > "Create Virtual Network"/"Details" tab > Expand "IPv6 configuration" > check "Enable IPv6" > enter "Network" prefix and/or check "Enable DHCPv6" > enter "Start" and "End" addresses

If you use a network bridge there's no reason to have to configure any IP addresses at all.

Yeah this has always been a bit of a pain. Since LXD recently launched a web interface that includes graphical console support (plus VM support some time before that, it's not just a container host anymore, it can do both), I have been using that for remote VMs now. But it still has quite a few rough edges, but nicer than libvirt at least.

virt-manager is the kind of software that frustrates me, has lost me VM (due to beraking SPICE beyond it already having been broken for years). Bug reports go ignored. Yet, when it works, it works great. And KVM is great. But Christ, it's a nightmare sometimes. Features lag behind libvirt, sometimes years. Just not great. The virt-viewer port to Windows never made it past POC+1, IMO, also.

I'm honestly surprised there aren't more alternatives. Especially since SPICE has gstreamer instegration. Gstreamer can natively do webrtc video and data back channel. Someone could hack together a webrtc broweser-based VM viewer and I suspect there's a lot of interest, potentially money making things you could crate.

So this basically turns VirtualBox into a replacement for virt-manager and virsh?

AFAIK VirtualBox does not support PCI passthrough (like GPU), how is this case handled?

Since I've moved to QEMU/KVM on Linux I've never looked back at VirtualBox, but I use the latter on Windows and there I'm always remembered of how much nicer and friendlier the GUI is.

Edit: I just noticed that VirtualBox has experimental PCI passthrough via the extension package [0], could this be used with the KVM backend?

[0] https://docs.oracle.com/en/virtualization/virtualbox/6.0/adm...

There is experimental support for VFIO PCI pass-through with VirtualBox-KVM, even for GPUs. Please have a look at https://www.cyberus-technology.de/products/hypervisor

You can use `VBoxManage --attach-vfio` if you want to experiment with pci passthrough. This is different from the Oracle `pciattach` call and does not require any support from the extension package.

Please evaluate the license of the extension package carefully before using!

And don't download it from your employer's network if Oracle thinks they might have deep pockets.

Practical thing is not having to recompile 3rd party drivers (vbox ko) every time kernel gets upgraded. Tho DKMS tries to take care of that without admin intervention, it's not always bug free.

On the other side VirtualBox the software application is designed to integrate with the desktop well, VNCing into the guest is not an alternative to this.

I hope efforts will be made in FreeBSD world too against its bhyve hypervisor.

The terminology issue is curious but it has been already covered here lately in a topic about Linux direct rendering manager, DRM. The acronym was used in a narrow circle of people compared to "the whole ICT", as were keyboard-video-mouse devices. Less than 1% of professionals deal with Linux internals on that level, and less than 1% of professionals are server room on-site engineers. There wasn't collective consciousness about these terms so they got reused.

Also LVM is taken by something else (storage) and LKVM would be confusing.

I cannot even imagine using a distro that can't manage to get DKMS/kernel module updates to be reliable. I don't think I've ever had a problem with DKMS in NixOS ever; not a single time.

edit: downvoting me won't make your distro any more competent.

dkms has worked fine for me in Gentoo, Debian, and Ubuntu.

I think the backlash has more to do with the comment's tone.

Doing this for FreeBSD would be a great project. Unfortunately it is also big enough that we couldn’t afford to do it without some kind of funding.

What does this give me that I don't already get from KVM and virt-manager on Linux? Not getting it.

A decent UI and hopefully support to use virtualbox appliances

The out-of-the-box performance of Windows in VirtualBox is very good and usually better than virt-manager (Qemu). You can tune Qemu to great performance as well, but it takes some fiddling. VirtualBox is in general very user friendly.

Guest integration (drag'n'drop, clipboard), USB passhthrough and audio support is also top-notch in VBox.

> The out-of-the-box performance of Windows in VirtualBox is very good and usually better than virt-manager (Qemu). You can tune Qemu to great performance as well, but it takes some fiddling. VirtualBox is in general very user friendly.

I haven't found a significative difference but if you have found one and can tune qemu to same level,why don't you share the xml template of your machine to the world and to upstream's virt-manager project?

> Guest integration (drag'n'drop, clipboard), USB passhthrough and audio support is also top-notch in VBox.

These things works well with libvirt too provided you are using the spice-guest-tools.

Not sure about drag'n'drop. Also I've noticed that even when you're aware of the way USB passthrough in virt-manager GUI works that it seems to have some bugs.

I'm mostly interested in if I can use virtualbox accelerated video with kvm because virgl3d seems well behind in that area.

ah yes maybe drag'n'drop is not working I have no idea tbh but I don't remember it working reliably in virtualbox and shared folders always worked better in my limited experience.

Shared folders does indeed seem like a weak point for kvm/virt-manager. There's the virtioFS but this is a pretty recent addition that was also recently pretty buggy on Windows.

I'm not even sure what your alternatives were for this before now, I guess everyone was just using samba.

Until you want to pass a GPU to the VM

With this version of VBox, it's quite possble to pass a GPU to the VM. Have a look at https://www.cyberus-technology.de/products/hypervisor if you want to see a demo.

I virtualize most of my desktop environment. I wanted to go with KVM and virt-manager initially, since I'm mostly using a Linux host and Linux guests, but there were two important features I wanted and couldn't figure out how to get that way: encryption and portability.

Most of the VMs are encrypted, so I feel safe traveling with them. Various secrets are also encrypted, but the encryption of the VMs themselves mean that I don't have to worry about losing my device at an airport and someone else potentially getting access to things they shouldn't. There are schemes that make this work in virt-manager and KVM, but I didn't like any of them as much; I didn't want to rely on the host for filesystem-level encryption (see portability), and I have previously had a bit of trouble with full disk encryption, so I wasn't comfortable relying on that. VirtualBox essentially is also doing full disk encryption, but it's invisible to the guest and seems to be reliable.

For portability, I should be able to use https://www.vbox.me/ to install the VMs and a host onto a flash drive and be able to run any of my environments from any Windows host without additional installations. Haven't actually tried this yet (happily, I no longer have easy access to Windows machines!), but it was a big point in favor.

Most of my environments now get auto-configured through Vagrant: https://github.com/robsheldon/vagrantfiles, so I get some of the benefits of virt-manager that way.

I really don't love relying on Oracle for anything

I have ever only heard KVM in the context of a Keyboard Video Monitor-type device but somehow I can't fully fit that into the concept of a virtual machine. Does it mean something different here?

https://linux-kvm.org/page/Main_Page

Is there a connection to User Mode Linux (UML) from around the same time? Or are these completely unrelated projects? I get that running a kernel in the user space provided by another kernel is not really the same as a proper hypervisor, but have never really dug deep into why and what the various tradeoffs are.

There is no real connection to UML here. Hardware virtualization (Intel VT, AMD-V) are much faster in practice and also don't require the guest operating system to be heavily modified. So besides as curiosity or test vehicle, approaches like UML are pretty dead.

Thanks.

So, slow as it may be, the win for UML (which seems to still have a heartbeat) is that it can run on uP without any specific virtualisation capabilities, right? If I could run Linux on a Z80/6502 then in theory I could run a virtualised Linux on a Z80/6502.

Yes - plus the original win of UML was also being able to run virtual instances on a kernel without proper virtualization capabilities.

In the early 2000s people used to use UMLs as a hosting platform - they didn't have the same security isolation as a proper VM (or even, necessarily, of a container) though.

How do containers have better security isolation than UML?

When I tinkered with UML I think it was prior to cgroups (2007) [0] so my guess is that escaping the UML instance was easier.

[0] https://en.wikipedia.org/wiki/Cgroups

The “original” UML is/was, I believe, NetBSD running as a “rump kernel” and something that virtualization of the actual kernel does not, directly and on its own, fill the shoes of.

UML is older than NetBSD rump kernels.

I stand corrected.

I want to know what Linux a386 was. Couldn't ever really understand what it did.

It means Kernel-Based Virtual Machine, a VM engine dating back to 2007: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine

Yes, quite different. It's basically a project that allows you to use Linux as a hypervisor. A very popular project.

https://en.m.wikipedia.org/wiki/Kernel-based_Virtual_Machine

Thanks!

Funny enough, a KVM might be needed for KVM if you give the VM control of a USB hub and a discrete GPU using IOMMU and you only have one monitor and set of peripherals, but two hubs and an integrated GPU.

Happy 10000 Day!

https://xkcd.com/1053/

While an over simplification, here is the context

There are two large hypervisors in the Linux world.

Xen, which extends the kernel to support virtual CPUs with time slices.

KVM, which assigns each virtual core a process that uses the Linux scheduler.

When a hardware vm vcpu core is preempted there is vmexit call that has to reset registers etc... and it is expensive.

Xen is what legacy AWS instances ran on and has advantages for being fair to guests is an easier task.

KVM has the advantage of gaining the benefits of the Linux scheduler which is red black tree based and well optimized.

When a new CPU comes out for example, KVM gains support from the upstream while Xen has to support it themselves.

Once technology like cgroups improved the benefits of letting your thread complete and not be preempted due to the time slice expiring avoided the cost of vmexit.

In theory, leveraging the inherently optimized core Linux features is what will also benefit virtualbox.

Most people who use KVM are using an abstraction layer like libvirt that hides how it is implemented.

In fact if you look at the processes you will see qemu even if KVM is how it is implemented.

I am really curious about this.

As a pentester, I run use Linux on my laptop and I spend a lot of time working inside a Kali VM with VirtualBox.

How much performance improvement can we expect with the KVM backend ?

It depends on your setup and workload. On a recent Intel CPU, our performance dashboard shows +10% for some benchmarks. It's hard to make a general statement though.

If you don't mind, I have a specific question regarding this setup. I've been looking into getting into pentesting, mostly for fun. I decided on messing around with HackTheBox as a starting point. Seeing as you need to connect to their VPN, it seems like a good idea to me to separate this activity from my personal network.

Which networking setup do you use for your pentesting VM? Ideally, I'd want a setup where the VM can access the internet (and therefore the HTB VPN), but not anything inside my local network. But I don't quite know how I could achieve that, at least in a way where I'd trust it to be reliable. Maybe the whole idea's a bit too paranoid to be practical in general, I don't know, so I'd love an expert opinion on this :)

Usually, I'd be using QEMU, but I'd be fine with using VirtualBox for this case if it includes something that makes this easier.

You can probably do this for VirtualBox (and any Linux program) by using tun2socks to create a network interface that routes through a proxy (SOCKS5 if you want UDP support), and then moving that network interface to a new namespace. You can run VirtualBox or any other programs in that new namespace, they don't have to be aware of the proxy at all (since they just see a regular gateway).

https://github.com/xjasonlyu/tun2socks

Which namespace are you taking about? cgroups_namespaces(7)?

Fancy. Think I'm gonna go with a traditional bridge for my usecase, but this looks pretty cool. Maybe I have some other uses for it for other applications.

Thank you :)

That's just a simple rule on your firewall. I don't understand why you think you need to do this though.

I always assumed that the VPN connection is shared by many HTB users, so it seemed like a good idea to be a bit paranoid about the trustworthyness of a VM that is reachable by lots of users that may want to mess around. Maybe I was wrong about that assumption :D

Anyway, apart from this possibly impractical usecase, it just seems like an interesting problem.

I'll just go with a standard bridge + ip/nftables setup then. Thanks for the input.

You could take a look at Whonix and Qubes - even if you don't use them you can roll your own setup with a similar approach: Have a dedicated "netvm"/"proxyvm" which only acts as a network bridge. "Minimal Whonix" consists of one workstation VM and one gateway VM (forcing everything over tor). Qubes integrates this approach into the UI of the hypervisor.

Then you can be more confident that (barring something escaping the VM), you have control of what a particular VM may or may not access. Also allows you to have a couple of different netvms (maybe one does give some access to your internal network, one connects only over tor, one via some VPN...) that you can switch between without needing to change anything on the actual VM of concern.

I've contributed in the past to libvirt in order to support some Virtualbox features because some of our customers used VBox. It would have been handy to have this in the past, and have all of our customers use some KVM VMs ;)

Congrats for the work!

Thank you!

This is cool! However maybe I'm just old and grumpy now, but this seems like something that would have been awesome 10 years ago. Now does anyone really care though?

When it comes to linux-in-linux virtualization of GUIs you can do much better than virt-manager/vmware/virtualbox nowadays. With crosvm you can share Wayland through shared memory buffers and obtain opengl & vulkan acceleration for the guest while at it. This way guest applications appear through the host's Wayland compositor seamlessly.

Yes, this is indeed nice. I have a Chromebook as well and the integration with Linux apps is super seamless. The major usecase for VBox is mostly Windows, though.

With this setup, what is providing the emulated hardware devices? (emulated USB host controller, emulated interrupt controller, etc)

Is it still the original set of emulated hardware provided by virtualbox, or is it now whatever KVM provides?

The emulated hardware is basically the same as with stock VirtualBox. Only the interrupt contoller (local APIC) is emulated by KVM.

Can someone ELI5 what this is and does it benefit someone like me who occasionally spins up VirtualBox VMs for various OSes?

This only affects anyone that wants to spin up VirtualBox VM's on Linux hosts.

I'm not an expert in this field, but my best TL;DR is that VirtualBox and other VMM's (virtual machine monitor) used to ship with their own hypervisors (the thing that let's you run virtual machines). However, now Linux has its own hypervisor/framework (KVM) and now VirtualBox can use KVM to do all the functionalities their own hypervisor used to do.

Someone please correct me if I'm wrong

Thanks! Appreciate it.

If you go into VM settings > system > acceleration > paravirtualization interface you can select KVM, what's the difference between this and that?

What you can configure in the GUI is an enlightenment that the guest will see. This is usually done to improve guest performance.

You can sill select these PV interfaces with VirtualBox-KVM, but the underlying hypervisor is different (kvm vs vboxdrv).

The paravirtualization option affects the interface presented for the guest operating system for dealing with being virtualized.

This new code is about using different virtualization technology on the host system.

I'm quite inexperienced with Virtualization. Are there benefits to kernel based virtual machines beyond (what I assume is the primary benefit) performance?

The name KVM is a bit confusing. It doesn't do anything fundamental different than VirtualBox. The difference is that KVM comes by default with any Linux. VirtualBox ships it's own Linux kernel module for that. That has drawbacks. You typically can't use the newest Linux or the newest features.

KVM also always has the hottest new (performance-relevant) features, because Intel and AMD will always build their hot stuff into KVM first.

You can’t use virtualbox’s kernel module and kvm at the same time. This basically means that you can’t use virtualbox and qemu at the same time.

If you use virtualbox with this new backend, you can use it concurrently with qemu (and a few other virtualisation tools).

KVM is also part of Linux itself, so there’s a lot less haste with setting it up.

virtualbox drivers are/were a constant source of kernel panics on MacOS and Linux too, so that should not be underestimated.

IIRC they also disabled ASLR kernel wide.

Additionally, and perhaps less important: USB3 is a commercial feature of virtualbox, there are stories of companies getting C&D letters (or Audits/Invoices) from Oracle because a developer had installed the virtualbox extensions..

Starting with VirtualBox 7, USB3 no longer requires the commercial extension pack but is part of the OSE release. See https://github.com/cyberus-technology/virtualbox-kvm/blob/de...

I wouldn't call it kernel based. It's not like this is an in-kernel emulation. I would stick with hardware virtualization.

KVM is a userspace API, and kvm-intel/kvm-amd are the drivers for the hardware.

You will be using hardware features. That's also why it is in the kernel: nothing but the kernel should have full unlimited access to the CPU to set this up.

So you could say it must be in kernel to keep the kernel secure. And the performance benefit is "just" exposed hardware features.

The kernel does not provide additional things. As far as I understand: you set up a dedicated memory space and handle traps that halt the execution e.g. when the VM talks to the PCI bus. (It's been a while since I looked this up)

But you need the pieces, especially virtual PCI devices. That's where qemu or VirtualBox enter the scene (or minimalist systems like firecracker). They provide a repository of virtual hardware and all the auxiliary methods to boot a virtual machines. You also need to emulate something like a BIOS or UEFI.

You can think of it as your CPU removing the need to emulate the very same CPU (and a memory controller). You still need to emulate the rest though! But running on the same CPU removes most performance penalties. You run at native speed.

Newer generations can even nest this. Having virtual machines in virtual machines. That's mostly useful for cloud environments so that the cloud provider can run kvm based VMs and you are still able to run VMs inside that VM.

That clarifies some things and gives me some tails to chace after! Thanks for the detailed response!

if in the past they already were using a kernel module then your reply doesn't explain anything.

So they went from using hardware virtualization (provided by intel/amd) with their kernel module to the KVM one. I don't know which benefits it brings

Without being at all up to date on the current state of things, the Virtual Box third party kernel module was historically of famously poor quality[1], even putting aside the general pains of third-party kernel modules.

[1] https://www.phoronix.com/news/OTk5Mw

there's no "keeping the kernel secure" and "allowing access to the hardware".

in security research, you either run your samples in qemu without even kvm or you don't.

Honest questions:

How does this work in licensing terms? If VB foss enough?

Do you expect Oracle to merge this?

If oracle doesn’t merge this, will you keep on maintaining it, potentially forking VirtualBox?

>How does this work in licensing terms? If VB foss enough?

It's as FOSS as the VirtualBox open source edition.

> Do you expect Oracle to merge this?

That would be nice, but I wouldn't hold my breath. Oracle gonna Oracle.

> If oracle doesn’t merge this, will you keep on maintaining it, potentially forking VirtualBox?

We don't intend to fork VirtualBox. VBox has a somewhat modular architecture where you can plug-in different hypervisor backends. That's what we did. It's not as modular, but our changes to core VirtualBox code is very small.

As far as our plans go, we are pretty open at this point. We are very interested to get to know people that find this useful!

This would be useful for anyone who needs to run bleeding edge linux kernels. Most other hypervisors have poor support when you're pinned to testing distros (For security reasons). KVM and virt-manager are uh not exactly user friendly, so being able to use the KVM backend and always be compatible with the new kernels while having the UX and UI of vbox is actually a very huge deal. This is one of the main reasons I really really hope you all manage to upstream this.

I built and tried this, it worked great, so excellent work there. I found the processor CPU core counts being grayed out unpleasant, it's not clear to me how I am supposed to adjust my core and ram values now, maybe document this?

That's odd about the core count. I only get that behavior if my host system only has 1 CPU to begin with (tested in qemu/KVM with nesting). Could you comment on your host system parameters a bit?

This is definitively not expected. You could also try setting the CPU/Mem configuration via VBoxManage. Maybe you get a good error message then.

`VBoxManage modifyvm <vm_name> --cpus <number of cpus>`

`VBoxManage modifyvm <vm_name> --memory <amout of memory in MB>`

User error, I had to discard the snapshot state first - just like in normal vbox ;)

Woah.. does this mean OVA/OVF support for kvm?

Yes.

Awesome! Building this now, super excited to try it out.

I'd love a realistic alternative to virtualbox on Windows for running Debian as a development environment.

I've tried hypervisor but I can't get networking to work, and trying to set up another adapter gives me a bsod on every computer I try it on.

Wsl2 almost works, but I can't really use Intellij to its full capacity. I'm under the impression that GUI application support is coming so maybe that will change. I'd really love for this to work so I can learn docker, but virtualbox performance is non-existent when wsl is turned on.

Qemu feels incomplete or at least overly complicated compared to Virtualbox.

This is awesome and great news, thanks a ton!

The biggest issue however is that many Linux distros just refuse to build and distribute VBox kernel modules despite them being open source. Thank you for your work regardless.

Do you intend VirtualBox to pick up your work and integrate it? I'm really looking forward to it. Have you already talked to the company?

I really don't want this to be a one off work to potentially become decrepit and unusable going forward.

Secondly, some VBox features belong to a separate closed source extension pack: USB2/3 support, PXE boot, VM disk encryption, webcam/camera support.

Will they work with KVM VirtualBox after installing the pack?

Thanks. You can indeed use the extension pack with KVM Virtualbox, just make sure that you don't violate any license agreements.

Why? How does this generate revenue for CyberusTech? What is the long term goal?

We're offering commercial support. We can also help with graphics virtualization and other topics (e.g. performance tuning and automated testing in real world scenarios).

Those are good questions. I don’t understand why you were downvoted.

To answer: We are offering service contracts and contract engineering services around virtualization, KVM and a couple of other topics.

The long term goal for this specific project is to support the security goals our customers have and to enable a couple of KVM features in Virtualbox as well. We plan to keep this updated with upstream Virtualbox and KVM.

Please upstream this code. That ensures this work will live forever with better maintenance without you all having to chase upstream vbox AND will be easier to justify use if it's an included vbox component. I would find it difficult to justify using this to my bosses "lets just go use this custom fork of vbox" isn't going to fly for most.

so, the plan is to take on docker hub?

edit: nevermind. i assumed you were a team inside oracle.

The blog post mentions an open source license but I can’t immediately see it in the post or the repo (perhaps I’m just missing it). Any idea what license this is released under?

It seems to be a fork of VirtualBox under the same dual license as the original project.

https://github.com/cyberus-technology/virtualbox-kvm/blob/de...

That appears to be the VirtualBox OSE license, copied from the original Oracle package, not the license for this specific release. It’s unclear how this new derivative or work is licensed.

The intention is to have this under the same license as the VBox open source release. If there is a way to clarify this more on the Github page, please advise. :)

Thanks for the clarification, that’s really helpful. I think a paragraph under a “License” header in the README just reiterating what you said in that reply would be pretty clear.

I’m sure some people would make the assumption that it’s under the same license as the upstream package but in some environments absolutely clarity around licenses is really appreciated.

Ok. We'll try to clarify the situation in the README. Thanks for the feedback!

Tested on ubuntu 22.04 with an amd 5800x and amd 6800 gpu. It seems to work flawlessly, the gui seems to respond as quickly as the desktop its run on, it even has the same timeout errors clicking through the "welcome to ubuntu" setup. This is my first time running virtualbox on my desktop/linux, so I am not sure what I should expect, but I certainly have nothing to criticize here.

I'd like to know more about this part:

> A lot of the security features that get built into Windows rely on virtualization. In cases where Windows is already virtualized, additional support from the Hypervisor is needed to enable those features. KVM provides a lot of the needed support to improve Windows’ security in virtualized environments.

What specific features does KVM provide for this? I gather this isn't talking about nested virtualization.

Any chance this gets upstreamed?

I mean it's Oracle we're talking about here.

"What you think of Oracle, is even truer than you think it is. There has been no entity in human history with less complexity or nuance to it than Oracle."

Bryan Cantrill

https://www.youtube.com/watch?v=-zRN7XLCRhc&t=1980s

Thanks. This is pure comedy gold. In particular, that part about the acquisition by Oracle (from minute 33)

Oracle already had an unfinished and broken KVM backend in the code that was not exposed. Whether they incorporate this polished KVM backend is anyone's guess at this point.

How is it broken ? I used it for a day recently (from ubuntu repo, windows guest) and it worked ok.

The KVM backend in vanilla VBox cannot be activated without changing the code. If you tried it, you tried the vanilla VBox hypervisor (vboxdrv) instead of KVM.

I’d actually much, much rather see it set up as a proper fork (rebranded etc.), and then the features that Oracle extorts people with in the “free” but not actually free extension pack (like USB pass through) re-implemented and included directly with no ‘extension’ required.

Much of the reason I refuse to use Virtualbox for anything is how scummy Oracle is.

Haha. I understand the sentiment. That's a pretty large effort though and needs some funding as well.

At a previous employer Oracle sent a nastygram because they saw downloads of the VirtualBox extensions pack (which is free to download, but requires a license) coming from our IP block. This despite the fact that we were a big Oracle customer (tons of Oracle DBs; granted, we hadn't purchased VirtualBox licenses). I'd rather not deal with a vendor that's that antagonistic.

> This despite^H^H^H^H^H because of the fact that we were a big Oracle customer

FTFY.

Having spent time at another large Oracle customer that was later acquired by Oracle I've seen how they do this internally too. And you can't just send it off to Legal to write back that "we didn't actually use it"...

This employer has a massive network with a bunch of guest networks (it's a hospital chain), so I thought they could just say there was no way to know whether it was employees or not. Of course, IIRC they just paid the ransom and added new restrictions on how employees could manage their work machines.

Can someone break this down for me? I gather this is not included in VirtualBox itself. Is it a plugin of some type? Is it useable?

Edit: Apologies. The answer is in the article itself: Compile VirtualBox with this Release from source[1].

[1] https://github.com/cyberus-technology/virtualbox-kvm

I think this should solve a current issue I have with virtualbox dealing with nested VMs. For example, nested QEMU VM running in a Virtualbox Linux guest causes the guest to lock up. According to virtualbox forum thread, only virtualbox-in-virtualbox nesting is supported, so to get around this I use virt-manager. But I think this kvm backend should solve it.

The KVM backend doesn't have nesting enabled just yet. We're on it.

Running VirtualBox-KVM in a QEMU-VM with enabled nesting should work though.

So can this run/import existing vbox VMs?

Yes, we are switching between vanilla VBox and KVM VBox during development quite often and the VMs are fine with it.

First time hearing about Cloud Hypervisor. What's the use for it? I recently built a Windows 11 Pro machine (mostly for gaming) but would like to run a few VMs on it. What's the recommended VM platform these days? Still Hyper-V? I'm a macOS and Linux guy, but begrudgingly using Windows because of games.

Hyper-V is a good option for VM servers but AFAIK it doesn't do user-friendly stuff like USB or graphics card pass-thru.

If I already have a headless debian hypervisor using KVM / QEMU in place running multiple debian VMs, can I now use Virtual Box to manage / tweaks the config on those? And if so is there anyway to do it without having to install a window manager, etc. on the hypervisor?

When Apple Silicon client? WHEN!?

I'm running a Ryzen5600G on Arch and recently switched from VirbualBox to QEMU/KVM and couldn't be happier.

Webcam, USB drives, Hardware Video Acceleration, all working without issue (after a pretty tough learning curve getting it set up).

Yes, I recently had to compile some stuff on Windows (I'm on an AMD Linux host) and VirtualBox just wouldn't start Microsoft's Windows dev VM (the one they provide for free for Virtualbox). I ended up learning how to use qemu and it works great...and as a bonus I was able to run a hackintosh (via https://github.com/kholia/OSX-KVM) and it works near flawlessly, which was something I was never able to accomplish with Virtualbox (granted I haven't tried in a few years).

I'm pretty happy with Qemu now, even if it's jsut a CLI interface. I was tempted to try the virt-* stuff, but honestly it seems like one more thing to learn so I'm going to hold off until I need something like copy/paste between VMs and can't figure it out in qemu direct.

Is bridged networking easy to setup? It's the main reason i use VirtualBox.

Is it possible to use virtualbox's accelerated video adapters with the KVM backend?

Yes. Don't expect too much though. 3D acceleration with Virtualbox is rather flaky (independent of using KVM).

Sure, VMware's is better but even that isn't really a replacement for GPU passthrough unfortunately. What I'd really like is to just get GPU accelerated video decode in the guests.

This makes me consider using VirtualBox again. Having to install and update a kernel module is annoying. Especially on some immutable/image-based distros like Fedora Atomic.

Does it support extra features from VirtualBox (machine snapshots, suspend&resume, multiple monitors, shared clipboard...)?

It does.

I do all of those things daily with virt-manager already. Except multiple monitors, although I believe it is supported.

Thanks, it seems to support it (even through it is a bit DIY). Will look into it.

That's impressive. I find QEMU and libvirt quite cumbersome, so this looks like it may be a worthwhile alternative now!

How does it enhance security and performance of virtualized environments compared to the standard VirtualBox kernel module?

Well, KVM is used by Google and AWS and others for their clouds. As such, there are a lot of eyes on KVM code. The vboxdrv kernel module that provides the same functionality in vanilla VBox definitely has fewer people looking at it. It also has anti-features, such as code upload from the userspace VirtualBox process to the kernel. This is also the largest security issue with vanilla VBox, because a lot of emulation code runs directly in the kernel.

From a performance perspective, it's a bit more complicated. KVM has support for modern virtualization features (Intel APICv, AMD AVIC, etc) that vanilla VBox lacks. You get these in the VirtualBox/KVM version. On the other hand, vanilla VBox emulates most devices in the kernel (see above). So SATA emulation in vanilla VBox is very fast compared to KVM/Qemu or KVM/VirtualBox for a bit unfair reasons. Modern devices, such as virtio or NVMe, are not as impacted by that.

tl;dr So the performance you get depends on your workload. If it's very interrupt heavy, VirtualBox/KVM will win. If it uses antiquated virtual devices (SATA), vanilla VirtualBox (with vboxdrv) will have an edge.

And could one swap between the two backends with the same VM image (.vbox +.vdi) to see which one gave the better performance?

Yes!

eBPF for in-kernel device emulation, then?

EDIT: That was a joke, but actually it is a thing https://www.youtube.com/watch?v=nTMls33dG8Q

What took so long? Isn't this like 15 years late? How do people even use the thing without KVM?

KVM is the one built into Linux. VBox has it's own module for hardware virtualization all along

Reminds me of the bad old days when I regularly had to open a crappy windows 7 in a VM because some minor piece of software would not run on Linux or on the browser. Thankfully I haven't needed to do that in years.

On a more constructive note, this might be really useful for kernel developers, and for big companies using desktop virtualization.

Crappy? Windows 7 was the last good Windows.

You don't like being plastered with ads and news articles about Trump every time you open your start menu??

I use Win10 and I don't get ads in my Start menu.

Is it maybe because I'm using Win10 Pro, not Home?

Nope, I started with Home and upgraded to Pro later and I've never had ads in the Start Menu or anywhere else. My machine came preinstalled though and whoever installed Windows obviously had a clue and turned all that crap off before they shipped it to me.

Yeah, any other Windows would have been worse. For me it was crappy because no-one wants to be maintaining a second hardly-used OS on their laptop. You randomly need it once in a blue moon, have to deal with two UIs with their conventions and feels on the same computer, and with copying files between them, and then and of course the first thing it wants to do is download tons of updates. Makes for a crappy experience.

As a note, the first character of your COPYING file in the repo seems to be typo-d. ;)

Ooops. Will fix. :) Thanks!

That's already b0rken in the Oracle sources.

Oops. I probably should have checked before mentioning it anyway. ;)

Does this enable you to use Hyper-V in a guest Windows VM?

Not yet. Nesting support is on our list. But the performance will not be great.

Can use vfio passthrough gpu like qemu ?

i have gpu use in vfio, success passthrough to qemu vm using virt-manager. Can i add vfio to virtualbox?

Failing to find the interest over the various libvirt based GUIs

Too little, too late. VirtualBox is completely obsolete. And the fact that it is owned by Oracle doesn't help.

VirtualBox, as absurd as it is, needs a proprietary, licensed "extension pack" for basic things like today standard USB2/3 drivers, encryption and webcam. I don't care about backends as long VirtualBox frontend has "Oracle" slapped on it. So, kudos! for perpetuating Virtualbox' existence!

A lot of this has changed in recent VirtualBox versions. USB3 and Webcam support is now part of the OSE release.

Reply on news.ycombinator.com