Comment by pmjordan | Hacker News Reader

Comment by pmjordan | Hacker News Reader

Parent Full thread

The macOS kernel (XNU) does not have any kind of containerisation support. I believe "Docker for Mac" is actually a Linux kernel running inside a hyperkit/xhyve VM, and uses Linux's container support.

Apple hasn't put much public effort into making macOS work well inside a virtual machine. Recent versions in particular, where the window compositor uses Metal for drawing, don't run well. Apple hasn't opened the API for writing GPU drivers, so there are no drivers for the virtualised GPUs provided by VMWare, Parallels, VirtualBox, or Qemu in their VMs. (The tightening of code signing requirements [see the 'library-validation' flag] even locked out nvidia from providing their own GPU drivers from macOS 10.14 onwards.) The only way to have GPU acceleration in macOS guest VM is to pass through a whole GPU at the PCIe device level. (I don't know if anyone has tried AMD's MXGPU tech, which allows GPU 'slices' to be passed to VMs, with Macs and macOS VMs; the GPUs that support it are a little too expensive for a weekend project.)

There are signs that Apple does use virtualisation internally; specifically, they've started adding drivers for virtio devices and one of Qemu's display adapters to macOS.

> "Docker for Mac" is actually a Linux kernel running inside a hyperkit/xhyve VM, and uses Linux's container support.

This is indeed correct. Your Docker VM lives at:

    /Users/$USERNAME/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw

It would be very nice to be able to use libvirt from my Mac in the same way as my Linux workstation. Or be able to migrate VMs between the two. They're both using the same-ish CPUs, after all! (Workstation has a "i7-5820K" vs the Macbook Pro's "i7-8850H")

It would be very nice to be able to use libvirt from my Mac in the same way as my Linux workstation.

I believe libvirt as such is supposed to work on macOS. Furthermore, recent versions of Qemu apparently support using macOS's Hypervisor.framework. So it seems like this should be doable, although I've not had a chance to try it myself yet.

Cool, thank you for the info. It's been a while since I tried it.

> The tightening of code signing requirements [see the 'library-validation' flag] even locked out nvidia from providing their own GPU drivers from macOS 10.14 onwards.

Library validation has nothing to do with drivers.

Library validation has nothing to do with drivers.

Depends on the type of driver. For GPU drivers, it is indeed a problem.

I'm going to assume from your comment that you haven't actually written many (any?) device drivers on macOS, so let me explain roughly how GPU drivers on macOS work:

- One or more kexts provide an IOFramebuffer implementation, plus an IOAccelerator object which listens for various types of user clients. The IOFB and IOAccel are typically clients of the GPU's IOPCIDevice object and handle its interrupts and memory.

- A variety of user space plugins (essentially dynamic libraries) are loaded into any process wishing to use GPU acceleration, with different plugins for Metal, OpenGL, and so on.

The kext communicates directly with the hardware and manages memory, but the user space plugin tends to deal with the higher level tasks such as generating command streams, compiling shaders, etc., and they talk to the kernel driver using the IOKit user client mechanism. The plugins also provide the back-end implementations for the app-facing APIs such as OpenGL, Metal, and OpenCL.

In recent macOS versions, WindowServer, the process responsible for drawing your macOS desktop, has had the library-validation codesigning flag enabled. This flag enforces that any code loaded into the process must be signed using a certificate from the same Developer Team ID as the main executable or Apple's own certificate. This means, no third-party plugins can be loaded.

This also means third-party GPU drivers can't work with most Apple apps and WindowServer itself, because they require aforementioned user space plugin.

For similar reasons, classic Core Audio plugins (not to be confused with Core Audio Server Plugins) are deprecated - they used to be loaded directly into the process using Core Audio, which no longer works on apps with library-validation enabled.

Drivers which are implemented entirely as kexts or which are loaded into a "server" user space process are not affected by this, which I guess is what you were talking about.

> I'm going to assume from your comment that you haven't actually written many (any?) device drivers on macOS, so let me explain roughly how GPU drivers on macOS work

I have written a couple toy ones, but nothing major and I'll admit I have no specialized knowledge of graphics drivers.

> Drivers which are implemented entirely as kexts or which are loaded into a "server" user space process are not affected by this, which I guess is what you were talking about.

Yes, I was talking about how anything loaded into the kernel does not have to deal with library validation (though there is still the check that it's signed with a valid KEXT certificate, which I assume Nvidia already has). I generally call the user-space portion of a driver that's loaded into applications to be plugins and keep them outside of the umbrella of "drivers" (which I reserve for the KEXT portion only) but in this case I think we both understand what's going on and are just disagreeing on what the terminology is.

> For similar reasons, classic Core Audio plugins (not to be confused with Core Audio Server Plugins) are deprecated - they used to be loaded directly into the process using Core Audio, which no longer works on apps with library-validation enabled.

These are plugins that implement AudioHardwarePlugIn, right? I think that they're being disabled by default anyways to get people to migrate to AudioServerPlugin, but they were deprecated way before library validation was even a thing because they inherited the sandbox of the application they were loaded into, which was already causing them to fail in odd ways.

I generally call the user-space portion of a driver that's loaded into applications to be plugins and keep them outside of the umbrella of "drivers" but in this case I think we both understand what's going on and are just disagreeing on what the terminology is.

Given that the kext is essentially useless without the userspace component (and vice versa), I don't think it's unreasonable to call the plugin part of the driver. Note also that it's installed in /Library/Extensions. This certainly also matches the user experience: they download the NVIDIA "driver" installer which happens to install a bunch of kexts and also the user space components at the same time.

These are plugins that implement AudioHardwarePlugIn, right?

Correct.

More generally, the IOCFPlugin mechanism, which paired user space components with kexts, is essentially dead for third-party driver developers thanks to the increasing lock-down of user space processes.

Reply on news.ycombinator.com