Comment by lima | Hacker News Reader

Comment by lima | Hacker News Reader

Parent Full thread

> KVM and Xen have a smaller attack surface and are known to have a better isolation model, but I'm sure there are still plenty of exploits given enough resources.

KVM - the low-level kernel-level hypervisor - has a tiny attack surface and has been audited exhaustively. It's unlikely to have critical bugs in it.

When people talk about "KVM vulnerabilities", they're usually talking about vulnerabilities in QEMU, which implements the actual device emulation. QEMU has all of the attack surface, deals with low-level data shuffling, and is written in C. Even worse, most stock QEMU-KVM deployments simply run qemu as root with no extra sandboxing or MAC like SELinux/sVirt. It's very likely that a bunch of 0days exist for those environments.

This is why many cloud providers use KVM-the-kernel-module, but an in-house replacement for QEMU.

Fortunately, there's a growing ecosystem of QEMU replacements written in Rust:

- https://github.com/cloud-hypervisor/cloud-hypervisor

- https://github.com/firecracker-microvm/firecracker

- https://chromium.googlesource.com/chromiumos/platform/crosvm... (the Chrome OS VM runtime which Firecracker was forked from)

Google's gVisor - the sandbox that App Engine and Cloud Run uses - uses KVM as well: https://gvisor.dev/docs/

With an emulation layer written in a language like Rust, the trust boundary is much better.

As for VirtualBox in particular - that one should not be considered a trust boundary. Nobody is seriously using it in production, and it's regularly featured in CTF competitions as a fun exploitation target.

KVM and QEMU maintainer here.

> most stock QEMU-KVM deployments simply run qemu as root with no extra sandboxing or MAC like SELinux/sVirt.

This is not true. OpenStack uses Libvirt for example, so on Red Hat/CentOS systems it will be hardened with SELinux by default. The same is true for virt-manager, which runs as either an unprivileged "qemu" user or as the user that invokes it.

Not all the code in QEMU is of identical quality, but the subset that is used by cloud providers is very mature and is also tested in Google's open source fuzzing cluster. Thanks to a student from Boston University who came up with some pretty cool techniques, we can cover devices effectively without having to teach the fuzzer about each device's registers.

> many cloud providers use KVM-the-kernel-module, but an in-house replacement for QEMU.

Two of them--certainly they are the big ones, Amazon and Google, but pretty much everybody else is using QEMU. For what it's worth, Amazon is also using QEMU on their old Xen-based instance types.

Firecracker is the only open source virtual machine monitor that cloud providers use in production apart from QEMU, as far as I know. It's a very interesting project and the people behind it have also started rust-vmm, a library of Rust crates for virtual machine monitors and device emulation backends. Several QEMU developers also contribute to rust-vmm, focusing especially on the abstractions needed to avoid unsafe code in virtual machine monitors. There was already an instance last year of problematic unsafe code in Firecracker that wouldn't have been there had it used rust-vmm's vm-memory crate, and in fact Firecracker has since switched to vm-memory. So there is plenty of collaboration between the authors of various VMMs.

> Thanks to a student from Boston University who came up with some pretty cool techniques, we can cover devices effectively without having to teach the fuzzer about each device's registers.

Can I ask who this was and is there any more info on their work? Seems important and worth naming them :)

Alexander Bulekov. You can find their work in the git log but I think there's no published paper yet.

> Many cloud providers use KVM-the-kernel-module, but an in-house replacement for QEMU.

Which are typically closed source, which is quite a shame.

If you're using KVM within a single organization to compartmentalize VMs, but they are all ultimately administered by the same people and toolchain, the threat is much less.

A lot of the extensive security precautions for qemu are intended for a hosting provider threat model where multiple random tenant customers may be using VMs on the same hypervisor bare metal, in which case you absolutely don't want them to be able to escape the vm or read each others' memory.

crosvm by Chrome OS is open source. And was even used / forked by Amazon cloud. It's Rust based.

Small correction: gVisor does not use KVM under the hood. It intercepts and filters the applications system calls, acting as a sort of userspace hypervisor.

It does, you can choose between ptrace and kvm in the config.

Didn't know it supports KVM as well. I though it only supports ptrace, as that's how it was advertised. This is cool!

What do you mean by kvm being audited exhaustively? Would be interesting to read about this.

It has been tested by the syzkaller fuzzing tool for years, and Google has done extensive search for vulnerabilities in the code (they did find some!).

Reply on news.ycombinator.com