Hacker News Reader

Comment by modeless | Hacker News Reader

Wow, you can use Rosetta to run x86 Linux binaries inside the VM on ARM? That seems pretty awesome. https://developer.apple.com/documentation/virtualization/run...

View on HN

I've tried using this a while back, soon after Apple released Ventura beta to developers.

I can say that documentation (and examples) are quite nice and easy to follow, I managed to start Linux VM with Rosetta support, and SSH, in 1-2 hours.

Geekbench numbers were quite close when running in VM (via Rosetta) and natively, they were only a few percentages slower in VM, so it stands to reason that computation heavy workload would run great.

However, using development tools regularly resulted in segmentation fault or crashing for other reasons (e.g. when doing `npm install`). I didn't dig deeper to figure out what are the reasons for this.

Additionally, real-world (from developer's point-of-view) workloads such as running web servers, building software, or running tests, were quite slow compared to running in an ARM VM.

I found that the Rosetta/Linux support worked best to run x86_64 containerized workloads, and I never experienced crashes working this way. In my experience, this is far more reliable than using something like debian/ubuntu multiarch support to install both aarch64 and x86_64 into the base system.

I ran x86_64 geekbench in a Linux/aarch64 VM using both Rosetta and QEMU binfmt_misc support and found that Rosetta outperformed QEMU by about 3x. The extra speed is welcome, but I do hope that all the dirty tricks Rosetta uses are eventually able to be used by both non-apple implementations of x86_64 emulation and on non-apple ARM CPUs.

Maybe I'm missing something, what are the actual commands for running a Linux x64 binary on macOS ARM?

You don't run the Linux x64 binary on macOS ARM. You run the Linux x64 binary on Linux ARM inside a VM on macOS ARM.

It seems overly complicated. Is there any benefit over using Docker?

I'm not sure what you think Docker on a Mac does. Docker is a Linux-specific piece of software, and the only way to use it within macOS is to run a Linux virtual machine, with Docker running inside the Linux VM.

If the containers you want to run with Docker are x86 software, that Linux VM either needs to be an x86 Linux distro running in qemu emulation of a full x86 machine, or an ARM Linux distro using the new (not yet released in stable macOS) Rosetta for Linux translation.

Well, kind of. Docker is a product, with official support for Linux containers on Mac (and Windows containers on Windows!). Docker for Mac comes with a Linux VM as a feature of the product; you don't need to install it yourself inside a VM (though that works, too).

It does sound like adding Rosetta binfmt_misc support would allow Docker for Mac to ship an ARM64 kernel/VM image instead of an amd64 one and benefit from some performance boost, but potentially at the risk of reliability/fidelity. The entire idea of Docker is that the kernel ABI is a (supposedly) stable interface, and even if your userspace changed around it, a Docker container would have its own userspace and wouldn't care. Running a different-architecture kernel and dynamically translating it necessarily means that there will be visible differences in the kernel ABI. Sure, you can translate those differences, but that gets you farther from the promise.

Sure, there is a VM in Docker as well. I'm sorry if it wasn't clear, I'm looking for the most straightforward way. With Docker it's a single command, it helps with documenting the steps for other coworkers. It feels slow though. I'm wondering if the new way with Rosetta is going to be better in performance.

The most straightforward way is to use a product. This article documents the APIs for those building those products.

> Docker is a Linux-specific piece of software, and the only way to use it within macOS is to run a Linux virtual machine, with Docker running inside the Linux VM.

That is entirely untrue: https://docs.docker.com/desktop/install/mac-install/

Hell, it’s even on Windows.

In all cases, the Docker daemon is running under Linux. The Mac and Windows versions are merely bundling up a Linux VM containing Docker with a frontend that's as transparent as possible, but still with Linux as a hard requirement.

Pretending that the Mac and Windows versions somehow aren't using Linux VMs behind the scenes is of no use to anyone. It's a convenience for users when they can get by with ignoring the VM layer, but a detriment when we see people start talking as though Docker for Mac is functionally different from a Linux VM running Docker, and start assuming that enhancements to running Linux VMs under macOS would be inapplicable to and incompatible with "Docker for Mac".

> In all cases, the Docker daemon is running under Linux. The Mac and Windows versions are merely bundling up a Linux VM containing Docker with a frontend that's as transparent as possible, but still with Linux as a hard requirement.

I really doubt that's the case if you run native Windows containers on Windows.

Thanks for pointing that out. I hadn't realized Microsoft had jumped on the Docker bandwagon to that extent; it's far enough from the topic at hand and from anything I'd ever use that I overlooked it.

So while there is in fact an exception to my previous generalization, there's still no cross-platform compatibility magic to Docker aside from that of virtual machines. If the container OS is different from the host OS (or a different version of the OS, for Windows containers), then using Docker is an instance of using VMs, not an alternative to VMs.

Docker can benefit from it by running 1000x faster if they would be able to leverage Rosetta.

In my benchmarks running Geekbench via docker, Rosetta performed about 3x faster than QEMU's binfmt_misc translation. On certain microbenchmarks it was 10x-50x faster.

Docker is equally complicated under the hood.

Yes, but it's a single command to run, without the need to learn the internals, if the use case is just to run the binary.

This is developer documentation, it's for people who want the internals.

For example, docker-desktop on arm-mac can run x86_64 images by leveraging binfmt-qemu within docker's VM. Changing that to use binfmt-rosetta should have huge gains (as there's specific hardware support to enable rosetta).

Yes it's easier to use docker. Hopefully this trickles down so that you get the benefits of this while not changing your usage at all.

You just run them after you/your distro registers a handler for them. It uses binfmt behind the scenes.

https://en.wikipedia.org/wiki/Binfmt_misc

(To be clear, this is for running Linux x86 inside Linux arm64 VMs on an Apple Silicon host)

You can do this in UTM, it's a GUI that wraps QEMU.

UTM can use both native Apple Virtualisation and QEMU virtualisaion+emulation. The QEMU one can't make use of the M1/M2 Intel memory translation acceleration the same way native virtualisation can, but it can make use of acceleration on ARM64 workloads.

Has anyone seen a guide to installing OpenBSD on M1 Mac using UTM?

No, but given that UTM is a graphical wrapper around QEMU would this be a worthy substitute? https://codeofconnor.com/running-an-arm64-openbsd-virtual-ma...

Thanks. I’ll give it a go.

Only on Ventura, which isn't out yet (and quite frankly, the pre-release version is still a huge mess)

Reply on news.ycombinator.com