Hacker News Reader

Comment by _joel | Hacker News Reader

Has there been a significant development? Been using it with https://mac.getutm.app/ for a while

I've also been using the Apple Virtualization option with Linux VMs in UTM and have had zero issues whatsoever. The biggest benefit to doing so in my opinion isn't necessarily the noticeably better performance, but the RAM usage. I've noticed that Linux VMs in UTM that run using the Apple Virtualization Framework use significantly less RAM compared to otherwise because the framework provides dynamic memory allocation. This makes it very well suited to running Linux VMs on systems that have less memory (e.g., an M1 MacBook with only 8GB of RAM).

The only thing I'd like to see added to the framework is the ability for nested virtualization (which is now available in the M2 chips, but isn't built into M1 chips).

> The only thing I'd like to see added to the framework is the ability for nested virtualization (which is now available in the M2 chips, but isn't built into M1 chips).

Is there an example of this working in any context? Until then I don't see how it can be considered available.

EDIT - Confirmed below it does work in Linux on metal - Props to the Asahi team!

Yep, works in M2 in asahi linux, but I'm not sure Apple has it enabled in their framework.

Thanks for the confirmation, pretty awesome the Asahi team is delivering features before Apple can on their own hardware!

Somehow I cannot get a Fedora 38 (or for that matter, Rocky/Alma 9) installer to boot with this option, it all works fine on QEMU, though.

There could be knobs to twist but UTM doesn’t expose any, so…

Isn’t it really hard to use though? Like you need decomposed Linux bits.

+1 for UTM. I use it to run an arm Linux distro that uses Rosetta for x86 binary only programs. Works like a charm and blazingly fast!

Perhaps I am reading what you wrote wrong but are you saying it can use Rosetta within an ARM Linux VM to run x86 applications in the VM?

Yes.

https://developer.apple.com/documentation/virtualization/run...

Well damn that is pretty cool. I’ve not used UTM or the Apple Virt Framework but gonna have to look into that now! Thank you for the link.

UTM also have their own docs on it:

https://docs.getutm.app/advanced/rosetta/

And FYI, this is also available in Docker Desktop for macOS, which allows better performance when running x86_64 containers compared to the older solution where Docker was using qemu.

Do you have any sources for that? I basically gave up on Docker for M1 since the qemu performance is terrible.

Last time I checked, Rosetta only worked for one-off binaries, wouldn’t really allow docker to run more x86 code inside.

It is used for individual binaries. The Linux kernel is arm, but the binaries running on it can be x86 running on Rosetta. This works particularly well with containers, which come with their own libc.

Search for Rosetta on this page

https://docs.docker.com/desktop/settings/mac/

I'm not sure what you're asking here?

Rosetta is a tool for running x86-64 binaries on an arm64 OS.

I think you're asking if rosetta lets you run an x86 kernel, to which the answer is no - the whole point of this framework is to support virtualization, e.g. the OS is running directly on the hardware. The moment the OS can't do that, there is no point in doing anything other than emulation.

Yes, I've used it with Terraform plugins by setting the arch to x86 and running via docker. Needed to do this as some plugins don't have arm64 versions.

downside is that there is no JIT-cache so it's much slower than Rosetta on Mac as it always does the translation on demand

On the run so apologies for not reading the docs you attached yet but I’ve been having a helluva time trying to cross-compile rust from an M1 for x86_64. Will this method help in this situation? I’ve tried running an x86_64 vm in emulation to compile rust bits it excruciatingly slow.

UTM is great in theory but I can't seem to be able to get directory sharing to "just work" and it's very frustrating.

Under the roots, utm can choose to use Virtualization.framework or QEMU (IIRC there should be a setting for that). It seems that Apple is actively working on this framework.

At first I thought this was being done to pump us up for new Mac Pro hardware that might be on the horizon, but then I wasn't sure, but now I'm also still not sure it isn't .. ;)

But, regardless, the subject of using a Mac to host a plethora of different platforms (in my case, for build-server duties) is indeed a fascinating subject. If I can get my target/builds set up on the next Mac Pro, whatever that is going to be, I'll be quite happy to do so .. especially if it isn't just doing MacOS/iOS builds, but whatever else my framework of choice (JUCE/Tracktion) supports (Android, Linux .. Windows .. etc.)

I think this is being done because the Mach microkernel was designed to run multiple OS's at the same time. I think Apple finally has enough developers to start pushing into new areas such as a framework to advantage Mach.

Virtualization is, of course, based on a very standard KVM-like API. There's not much Mach in its design at all.

Mach is down below. This is what Mach is for the BSD that was originally part of the DARPA deliverable was just to prove that Mach could run on OS.

There's not much down below it. It calls into the Hypervisor APIs, which directly touch platform virtualization features.

We are obviously talking past each other. On MacOS Mach provides all the resources. Mach was designed to abstract Operating systems from the underlaying hardware. It could also run more then one operating system at time since that was normal expectation.

Right, but that has nothing to do with the Virtualization framework.

"Being done", I'm not sure anything is happening here other than a happy user pointing out that Apple have some advances in this area, but .. your point is that this has been there all along.

Alas, if only there were the hardware at scale which supported Apples' Mach implementation, at favourable economies.

One way this is favourable to this dev is if Apple finally give me a machine which I can use to Build All the Things™, including iOS and MacOS and Windows and Linux and Android, oh my!, virtually and/or with x86/ARM cores onboard, oh yeah!, without having to resort to the typically draconian bollocks on other platforms, or rent someone elses hardware, or whatever.

A new Mac Pro that can be loaded to the gills and host several competing OS's at once, for the purpose of software development? Sign me up, because I'm tired of build-box'ing rigs and fighting that darn license ..

When Mach was defined it was considered a centralized campus computer that could run all the needed operating systems on top of centralized memory and processing using IPC.

https://en.wikipedia.org/wiki/Mach_%28kernel%29

Sure, Mach is great. No argument from me here.

Well except that, a new developer-focused Mac Pro with that centralized campus feeling, sounds great to me.

I can only wish it would have both x86 and ARM cores onboard, though, to get that true all-round Mach prowess...

I build windows and Linux and iOS and macOS all on my MacBook Pro.

Mach was designed to run multiple OS interfaces in the same way that Windows NT was - multiple compatibility layers on top of a common kernel (e.g. WSL1). What is being discussed here is virtualization, for which Mach is no more suited than Linux, Windows, Sys V Unix etc.

Mach was designed to make clean separation between operating systems and hardware. Notice how easily Apple moves between hardware? The unsung hero is Mach and the design patterns that NeXT/Apple framework built on top of it.

That is largely due to a mix of Apple's investment in emulation technologies, and Apple's ability to exert influence on third party developers and vendors to keep up with their latest tooling. Very little to do with the Mach kernel.

The ability to move between hardware platforms also has very little do with the ability to run operating systems in virtualization. Case in point is that Apple is one of the last major OSes to include a hypervisor in the OS.

I'm certainly not an xnu specialist but from what I understand, you've basically got a BSD kernel (a "macrokernel") running in privileged mode that (roughly) just has a few services replaced by or connected to Mach ones. So I'm not really sure they get much portability benefit specifically from Mach versus "more traditional" kernels.

It’s easy from the user’s point of view. It’s not easy to write Rosetta.

Every modern operating system was designed to easily move hardware. (It also helps a lot that all hardware is basically the same thing.)

There is nothing special about Mach/xnu here and it is not helping anything.

I understand that you can use it with https://developer.apple.com/documentation/virtualization/run... ? What is the difference or improvement about using UTM (macOS newbie here :-) )

In the UTM page it says "Under the hood of UTM is QEMU, a decades old..." so are we talking about the same thing as Apple Virtualization Framework. I am confused qbout the QEMU mention.

There are two frameworks, Hypervisor.framework which is a low level abstraction of the processor's virtuazionation functionality, and Virtualization.framework which is basically a QEMU implementation.

I am confused by the different comments here: does UTM only uses QEMU or it also uses AVF if the chip architecture of the host and guest is the same?

Edit: I think the catch is that QEMU uses AVF if available [1].

[1] https://medium.com/code-uncomplicated/virtual-machines-on-ma...

Yes UTM uses HVF via QEMU, some comments above say it can also use Virtualization.framework.

We'll find out tomorrow I guess

We'll see :)

How does one actually enable AVF on UTM?

I've got some Ubuntu VMs set up already, and as per [0] I cannot find an option called "Use Apple Virtualization" neither in UTM, nor in the VM settings.

I am on Intel Mac.

[0] https://www.jamesarmes.com/2022/12/linux-mac-utm.html#once-m...

I think it's only available when creating a VM, you cannot change after the VM has been created.

Does this compete with Virtualbox, VMWare, etc?

Does virtualbox compete with anything? Every time I've used it it's dog slow and / or has network issues.

UTM doesn't have as many settings as VMWare Fusion but stability and performance wise it's on par. It's also open source and free.

But it doesn’t support GPU passthrough? Which seems like a pretty big deal for many uses cases.

Reply on news.ycombinator.com