Quick Look at Rosetta on Linux | Hacker News Reader

Quick Look at Rosetta on Linux

threedots.ovh

172 pointsgok41 comments

For context: Apple provides Rosetta for Linux to allow native x86 binaries to run inside Linux guests on Apple silicon hosts as discussed yesterday [0]. This article explores what other uses can be had for Rosetta on Linux.

[0]: https://news.ycombinator.com/item?id=31644990

Thanks to the ARM version of the Microsoft Surface, there's an ARM port of Windows 10 via Insider Preview Builds. What this means though, is that I can run x86 Windows binaries on my M1 Mac though Parallels Desktop (haven't tried other VM programs such as virtualbox).

Unsurprisingly CPUz has a bit of trouble running and is unable to give much information, saying the CPU name is Apple Silicon, the spec is ARMv8, and the clock speed of 3.0 Ghz, but most everything is blank.

> For context: Apple provides Rosetta for Linux to allow native x86 binaries to run inside Linux guests on Apple silicon hosts .... Thanks to the ARM version of the Microsoft Surface, there's an ARM port of Windows 10 via Insider Preview Builds. What this means though, is that I can run x86 Windows binaries on my M1 Mac though Parallels Desktop (haven't tried other VM programs such as virtualbox).

What a time to be alive! Imagine reading this in 2015 :D

I ended up having to buy a cheap Windows laptop anyway because while I could get the x86 app on this page to run under ARM Windows, I wasn't clever enough to get the Silabs driver to work:

https://hdfury.com/product/integral-2/

What really boggles my mind about Rosetta is that it correctly handles x86_64 instructions produced at run-time. So, for example, the Clozure Common Lisp compiler for the x86 works under Rosetta. Even the IDE works, which really blows my mind.

There is one thing that doesn't work, and that is code that relies on floating point exceptions, which Rosetta does not correctly emulate. It's a curious oversight for a system that is otherwise beyond impressive.

Something also cool that's not widely known: Rosetta emulates i386 instructions. CrossOver/Wine uses this on macOS to run 32-bit Windows EXEs.

I haven't tested yet whether this functionality works on Linux.

Cool! So does that mean I can run 32-bit x86 apps on a Mavericks VM on an M1?

There's another haiku lying around:

    $ cat /System/Volumes/Data/Library/Apple/usr/share/rosetta/rosetta
    translation is done 
    as preferred long in advance
    rosetta idles

I'm really giddy to see this being made use of by Docker / podman as the currently used QEMU based solution is slow as molasses.

In our case, the moment the M1 Macs came out, I have re-built our development images on ARM, but there are a few dependencies to proprietary software which is only provided for x64 and which I can't really support on ARM Macs ATM because running affected services through QEMU is no fun, especially as those are mostly about handling bulk data living in proprietary databases.

But if this comes close to what Rosetta can do on macOS, using it for those cases becomes feasible.

Really neat that it actually can be made to run on non-Apple ARM CPUs! I did not expect that.

FEX is the open source equivalent: https://github.com/FEX-Emu/FEX

Is there a good article explaining what is shared between ARM processors (ie. M1 / Graviton), what makes them "compatible" ? and how they can add specific features still keeping them compatible ?

> Is there a good article explaining what is shared between ARM processors (ie. M1 / Graviton)

No idea, but I found this comment from last year which is interesting and goes towards answering your question: https://news.ycombinator.com/item?id=27077545

ARM defines the Instruction Set Architecture for ARM-compatible CPUs. It has major versions, like Armv8-A and Armv9-A. The major versions have minor subversions, like Armv8.2-A and Armv8.5-A. The versions and subversions have mandatory features, which must be implemented by everybody who claims to make an ARM CPU, and optional features.

Like on x86, where there is the CPUID instruction, on an ARM CPU it is possible to determine what version and subversion of the ISA is implemented, and which optional features are implemented, by reading some special registers.

It is possible to make a program that on newer CPUs will take advantage of some features, but on older CPUs it will have a fall-back code path of lower performance.

Because Apple writes software only for their CPUs, they do not bother to write programs that will also run on CPUs made by others, which may lack some of the features implemented on Apple CPUs.

As mentioned in the parent article, Rosetta needs FEAT_FlagM, which is implemented only on ISA versions of Armv8.4-A or newer.

All the ARM cores licensed from the ARM company with the exception of those introduced in 2021 (and also NVIDIA Carmel and the last cores designed by Samsung before giving up), support only Armv8.2-A, and those older than Cortex-A55 (e.g. Cortex-A73, Cortex-A72, NVIDIA Denver) support only Armv8.0-A.

This leaves very few non-Apple CPUs which can execute Rosetta, i.e. only the latest smartphones of 2022 with Armv9.0-A CPUs (Cortex-X2, Cortex-A710, Cortex-A510) and the Graviton 3 servers with Armv8.4-A CPUs (Neoverse V1).

Besides the ISA level compatibility, there is also the problem with the standard peripherals, e.g. timers, which also may differ between ARM CPU implementations.

As explained in the parent article, Rosetta is not written to work with any timer frequency, but accepts only a few values, so it had to be tricked to work on Graviton 3.

Because on any ARM system it is possible to determine the frequency of the timers, it would have been possible to write Rosetta to work with any frequency, but again, there was no reason for Apple to make it more flexible, when they want it to run only on their hardware.

Note that ARMv8.6-A onwards mandates 1GHz as the frequency for the CNTFRQ_EL0 timer.

This is the quality content I come to HN for, thank you

Would this just allow to run x86 executables inside a Aarch64 VM? Or would the Linux kernel in the VM also be a x86 version?

If the Rosetta binary is only available after a certain remote fad path is mapped, I would guess the kernel needs to be native Aarch.

In the sample code, the buffer into which the key is read appears to be undersized.

Yeah looks like a mixup of 45 and 0x45?

I wonder if it will make it possible to run x86 Linux executables on ARM FreeBSD, same way you can run ARM Linux executables on ARM FreeBSD.

I wonder how much performance can still be squeezed on linux JIT/dynarec emulation if support is helped by the kernel itself.

what does this do? i mean can we run x86 programs on say raspberry pi?

what about oracle cloudtron alwaysfree vps?

> i mean can we run x86 programs on say raspberry pi?

No, the BCM2711 is only ARMv8-A, Rosetta needs ARMv8.2 or later

sure. not the pi itself but other similar devices which have that processor clas?

Sort of? You'd also need a way of guaranteeing total store order semantics for memory otherwise you may get random errors. I guess restricting the app to a single core might reduce that error frequency a bit

> I guess restricting the app to a single core might reduce that error frequency a bit

Shouldn't it eliminate it entirely? I'm pretty sure ARM guarantees sequential consistency on a single core, so if the app is limited to a single CPU (e.g. using taskset), there should be no issue.

The real problem is, performance will be degraded.

> The real problem is, performance will be degraded.

Compared to qemu performance will be awesome.

I'm skeptical, but optimistic. qemu user emulation with TCG is also a JIT compiler, is pretty decent, and doesn't have the "needs to be limited to a single core" when ran outside macos hardware, though it certainly has a lot of room for optimizations. I'm waiting for some benchmarks comparing the two.

I don't remember where I read about, but didn't Apple specifically designed their processor to make Rosetta run faster?

You’re probably thinking of having a Total Store Ordering mode that can be switched on and off on demand. Most ARM CPUs don’t have this feature since it isn’t necessary, but it’s the normal way that x86 CPUs work. Without doing it in hardware An x86 emulator/translator needs to use a ton of memory fence instructions to emulate TSO.

You can already do x86 emulation on linux ARM devices via other means[0] including on the Raspberry Pi. Keep in mind performance will be greatly degraded, but its possible.

[0]https://github.com/ptitSeb/box86

To me this all sounds like Apple is ready to license their CPU, let's hope!

>To me this all sounds like Apple is ready to license their CPU

Not at all. Apple only makes HW for the Apple ecosystem an that's it. Just look at everything they sell from monitors to earphones. Their functionality is crippled when not used with Apple HW because they're dependent on the Apple ecosystem to run at their full feature set.

What makes you think they would suddenly reverse that? They're just not in the business of building and supporting non-Apple ecosystems.

Except when there's additional money to be made. Apple Music is available on Android because that makes Apple money. But iMessage isn't because that would prevent some tiny bit of iPhone sales.

I suspect Apple Music is on Android for the benefit of Apple Music users who have multiple devices, some Apple some not, and want a music service that works across all of them.

I use a Mac but an Android phone. I'd really like if things like AirDrop or FaceTime or other macOS/iOS integrations worked with Android the way they do with iPhones. I had to reverse engineer Google's Nearby Share protocol and build a (barebones and buggy and incomplete) file receiver for macOS to put an end to my misery of transferring things, primarily pictures, from my phone to my computer. But obviously Apple isn't interested in any of this; they'd rather tell me to just use an iPhone, since that would get me more hooked into their proprietary ecosystem.

I guess it's a matter of weighing the potential upside of selling you on more Apple products versus the downside of making Macs and iPhones less functional (w.r.t. interoperability with other devices).

They built the tools to allow performant x86 docker containers. That's a huge pain right now. Probably in two years it'll be solved.

It absolutely would not surprise me if Apple were running a custom Linux distribution on a range of not publicly available ARM server platforms as part of their development chain. I suspect this is as much for their own benefit as it is for everyone else.

I’m sure this is correct. I’ve seen comments on HN multiple times about what they use Linux for in house. You can tell to some extent just from job postings too.

Somewhere I read that Apple use Linux (there own internal version) as the first target when developing their silicone, before then targeting macOS once development has progressed to a certain point. I can't find the article now though, surprising hard to google for.

EDIT:

It was a HN thread:

https://news.ycombinator.com/item?id=30729731

"I find it crazy because you know they developed all this hardware with Linux. Just release the darn code."

"Apple does bringup with Linux because the silicon team doesn’t necessarily want to deal with getting the Darwin team involved during validation."

Worth reading the whole thread, some interesting insights and insider(?) info.

Reply on news.ycombinator.com