Asahi Linux for M1 Macs: progress report for September 2021
asahilinux.org
asahilinux.org
Stuff like this makes me wonder: why does Apple do this? If I try to give them the benefit of the doubt, I can assume that these changes are done for performance, cost, power-saving, or maybe even security reasons. Otherwise it just seems like Apple does these things in order to make it harder for other OSes to run on their hardware. Which is certainly their prerogative, but it just makes me think less of them.
On the other hand, this is really cool:
> However, Apple is unique in putting emphasis in keeping hardware interfaces compatible across SoC generations – the UART hardware in the M1 dates back to the original iPhone! This means we are in a unique position to be able to try writing drivers that will not only work for the M1, but may work –unchanged– on future chips as well.
... but cynically (or perhaps just realistically), I can easily believe that this isn't done for reasons of openness, but because this makes maintenance of macOS itself easier for Apple.
So the scenarios that remain are: fingerprint to unlock device to boot (need to do some crypto before the OS, unless you want the fingerprint to just flat-out give up the key to anything sniffing the SPI bus), or somehow resisting data modification without requiring the user to type a password. I feel like Bitlocker tries to do the latter, but I don't know what attacks they are trying to protect against. (It's on by default on new laptops, but you can just sniff the key over SPI when the OS is booting, so what security does it actually provide?)
I'm guessing they're protecting devices against APTs: state-level actors with lots of funding, competitors intent on discrediting their ecosystem, NSO Group, etc.
As a side benefit for users and Apple it makes the entire chain more difficult to introspect/attack.
The OS now prompts to grant file access permissions to applications.
The NVMe design makes sense in the context of their common ASC architecture and how their SoC works. Various quirks are due to things like supporting their storage encryption. Even for things which we can't quite explain, I have no trouble believing that it made sense for them for whatever reason.
As much as many of us want to attribute positive or negative reasons or motivations to things companies do -- especially secretive companies like Apple -- it's a nice reminder that most decisions are made without malice, because they make the most sense based on the requirements at hand.
Having control over the entire hardware and software ecosystem means that there's no reason to follow standards to the letter when those standards get in your way and make things harder, more expensive, or even just not possible.
Anyhow, just wanted to finish by saying all of this work is truly impressive. Obviously there's still a lot to be done to support everything to the same level as, say, a Dell XPS machine, but the progress made so far is pretty amazing, and even though I have no plans to buy an M1 Mac any time soon, I'm always excited to read these updates.
A lot of people try to attribute some human aspect to a large multinational (as they are mostly just sociopaths controlled by shareholders in a sense), but technology-wise it's often just "it made sense for us at our scale".
Same with the weird USB-C PD controllers that are 'almost normal' but then specialised for Apple; it's not that they want to make it hard to repair, use custom software with or patent it and screw with other companies.. it just makes sense to change that part instead of changing another part. At scale, that is a choice you have, but also a choice you must make in such an implementation. This is of course not exclusive to Apple, a lot of the really high quantity mass produced electronics contain slightly modified versions of existing parts because that turns out to be cheaper/more reliable/better fit-to-spec than modifying the rest of the design around an existing part.
This is one of the baffling things about calculators or toy electronics (for light and sound effects) in the same vein; modifying an ancient chip with very crude software on a single-sided extra thin older style PCB material with no silk screen and only partial solder mask with a die-on-PCB with wire bonding and epoxy blob... all to make the assembly 3 cents instead of 4 cents, and reduce the lack of connections to make that 3 cent assembly have less possibilities of defects and causing less of a multi-thousand-cent service process to be activated (swap in a store, callcenter, website, email etc). Every time you get one less customer to call about a crappy firetruck where the lights stopped flashing in a 3 cent assembly is a huge win. This is of course an extreme example, more complicated hardware and software extrapolates quite extremely from there.
Oddly enough, some of those practises are implemented at a much less impactful scale in HP and Dell desktops where they might have one large non-standard PCB hosting all the normal PC components but also all the DC-DC converters, front I/O and on-board WiFi. It makes almost no sense at all to do that, but the reduced number of connectors apparently makes the products cheaper to support, last longer and since people don't upgrade them anyway they are often just 'used up' and thrown away. That last part is bad of course but something that a design-for-manufacture department is unlikely to care about or mitigate using a trade-in or recycling program (which often just ends up meaning: ship the trash to china or Africa and let them deal with it).
The amount of details and their impact at this scale is astounding. Add custom silicon and you're almost in a new dimension.
Keep in mind too that "requirements at hand" can include a significant degree of path dependency, and that it can be a mistake to read too much reasoning into something too. Sometimes there just isn't any master plan, some decision made years earlier has created a dilemma due to dependencies built on it since and there just isn't the resources (or ROI, particularly without any spec compatibility to worry about) to deal with it right then. For a hardware company with necessarily very long lead times (they can't exactly start having chips fabbed and making electronics two weeks before launch) Apple runs a pretty hectic schedule with major new launches every single year. Even a vertically integrated company on their scale is not completely immune to the challenges of tight coupling or thinks of it all ahead, and there are definitely decisions Apple has made that they regret but can't easily get out from under. For example, just a few weeks ago HN had a sizable thread on "Swift Regrets" [0] by Jordan Rose:
>"I worked on Swift at Apple from pre-release to Swift 5.1. I’m at least partly responsible for many things people like about Swift and many things people hate about Swift. This list is something I started collecting around when I left Apple, and I’m putting them up so other language designers can learn from our mistakes. These are all things that would be hard to change in Swift today, because they’d break tons of people’s code. That’s what happens with real-world languages and libraries: the more users you have, the fewer breaking changes you can make."
For Apple same definitely turns up in hardware once in a while too, though of course they try to be careful and have a lot of institutional knowledge about pitfalls by this point. Can't always look at something they're doing now and assume that if they were greenfielding it they'd do the exact same thing again.
That's part of why they've long been (even before iOS) such hardasses about stuff like 3rd party use of private APIs under development. They certainly don't have any active incentive to break existing stuff, if everything could magically work forever that'd be awesome, but simultaneously they know they fuck up sometimes and want to be able to make changes. Hard to balance those in software without some kind of push to devs on putting experimental OS frameworks into production software or spraying all over the drive for installs. Users will inevitably come to depend on it anyway and now you're stuck.
For hardware I think it's obscure enough they're not worried about it, as absolutely awesome as it is Asahi Linux is never going to pinch them there.
----
This is why I'd say there's some value in sticking to spec even if it doesn't 100% meet your needs or do things the way you think they ought to be done. Tacking away is as likely to screw you in the long run as it is to benefit you. The benefits would be much more obvious, of course, in a world that's almost an unimaginable utopia from our perspective: one without intellectual property, in which major advances like Apple's ARM silicon were widely shared and put to the good of humankind in general, rather than held privately for the profit of a single corporation.
That sounds eerily similar to a paperclip maximizer (https://wiki.lesswrong.com/wiki/Paperclip_maximizer). I suppose we all know what Apple is maximizing, but I never drew this parallel before.
That's my read on this as well. They designed it so it would work well with their other hardware/software and this is what they landed on. I seriously doubt they paid even a second of thought to other OSs running on their hardware. Some people will call that "user-hostile" but it's much closer to apathy IMHO and as a macOS and Apple hardware user I'm fine with that. It's cool that Linux can/will run on the M1 but I'll never be doing that myself. It reminds me of that scene in Mad Men "I don't think about you at all" [0] but Apple's feeling isn't even as sinister or hostile as the comment made by Don.
Exactly this. I can easily imagine somewhere in some OS slack channel in Apple:
hardware dev: hey dear @channel we're breaking NVME spec in couple places and we're kinda running out of time to fix it, would be so kind to address it in you drivers if it's not too hard?
os guy: yeah, no worries those seem trivial, we can easily adjust.
hardware guy: cool, thanks a bunch.
That's naive.
It is absolutely part of their design goals to plan for obsolescence and to prevent third party OSes to run perfectly on their device. The move to soldering SSDs and RAM on laptops and desktops is designed to prevent users from extending the life of the device. Mac Mini's experience a delay of 2-3 minutes to even start booting alternate OS to frustrate the users. The T2 chips even prevent booting a lot of other OSes. There are umpteen examples like these.
And that's all ok when you consider the business goals of Apple. But please don't pretend that it's all an "accident" or Apple doesn't care if a user wants to break free from the stranglehold of the Apple ecosystem.
The reason for most of these things is most likely that Apple didn't start from scratch with the M1. M1 Macs in many respects appear to be an exercise in "how can we make our existing iPhone / iPad system architecture into a general purpose computer", and so comes with a surprising amount of legacy idiosyncrasies. For example, Apple started using NVMe-like storage all the way back in the 2015 iPhone 6S, and therefore was 1) very concerned with fitting everything into a small, power efficient package and 2) not very concerned with being standards compliant at all.
If Apple started from scratch with the M1, it would have most likely been more standards compliant. Out of Apple's self interest and ease of maintenance, not to help the community. If there's anything Apple has shown - on Macs only - is that they're not really hostile with respect to standards compliance and helping third party OS support. It's just that they don't care _at all_ about supporting that, and prefer supporting their own internal processes every step of the way.
(Not an expert on this topic at all, just echoing my impression of the system architecture choices)
Apple wrote Boot Camp drivers and the Boot Camp Installer for Windows on Intel Macs. The Boot Camp Assistant itself also paves the way for users to install Windows, for example, by partitioning storage for Windows installation.
The Linux community had to reverse engineer Apple T2 drivers itself though. Apple didn’t restrict anything there, but the community had to figure it out on its own.
UART as in a simple serial debug interface?
I thought they are fairly simple - isn't that like celebrating Apple for having the same transistor design since some date?!
On the other hand if the respective authors like accepting such challenges and working on these projects, there is nothing wrong with that. :)
I’ve been struggling with this in Lunar (https://lunar.fyi) for a long time and while I tried my best by comparing ioreg dumps and looking at the DCP driver in IDA, I couldn’t find any obvious logic would block this communication voluntarily.
I should mention that on M1, the DDC messages are sent to the monitor by calling IOAVServiceWriteI2C on the DCPAVServiceProxy of the monitor as seen here: https://gist.github.com/alin23/b476a02a8cd298436848e28476aed...
I’m thinking that this logic might either exist in the DCP firmware which is not accessible from userspace, or it might just be a side effect of some out of spec behavior that the HDMI port might have.
The HDMI port on the Mini is funny. The M1 supports exactly one internal display (the panel on a MacBook or iThing) and one external display (over DisplayPort/Thunderbolt). This is why M1 MacBooks can only drive a single external monitor.
For the Mini, the "internal" display is an internal DisplayPort connection, converted to HDMI with a mcdp29xx chip, and stuck on the HDMI port. Expect weirdness.
So its possible that it isn’t DCP related after all, the mcdp29xx chip might not be implementing the DDC part at all like most USB-C hubs that I have to deal with.
Thanks for the insight!
I don’t have an M1 Mac Mini to do more thorough tests, but one Lunar user reported that the DDC set brightness command worked for a short second on the initial connection of the monitor.
So from that I’m guessing that the MDCP29xx keeps the DDC channel open until it gets the EDID and does whatever handshake is needed and then.. maybe it closes the channel?
Another weirdness anecdote is a user which reported that the DDC commands sent to the Thunderbolt-connected monitor were also sent to the HDMI monitor at the same time, but (as usual) writing directly to the HDMI monitor service did not do anything.
Could you maybe point me to where I should start looking for the MDCP29XX assembly? I previously looked at `/System/Library/Extensions/AppleMobileDispH13G-DCP.kext` but I’m thinking that what you’re talking about might not be a kext.
Awesome job, love the enthusiasm.
I wish he was doing tutorial style videos, too. Pleasant voice, well-spoken and incredible knowledgeable. I bet he could do videos which don't have you thinking "Yeah, but why does this work?".
Thank you very much for doing what you do! You are an inspiration to me and I admire the calm and structured way you work. Hope you keep enjoying what you do!
And the concept of “free” time is also relative. For example, you could argue that child rearing is a game for those with lots of free time.
However, Apple is unique in putting emphasis in keeping hardware interfaces compatible across SoC generations ..... the device tree then can be used to represent the dependency relationships between these power domains dynamically. ... This approach is unfamiliar to most upstream subsystem maintainers, but we hope they recognize the benefits over time. Who knows, perhaps this will inspire other manufacturers to do it this way!"
This is really weird commentary to me, as far as I know Device Tree has been the standard for embedded ARM drivers in the Linux kernel for years, and for several years on PowerPC before that. When I worked in embedded linux, often the "bringup" for components in new ASICs was to create the device tree definition. What am I missing here?
So, a GPIO driver for a random SoC might hardcode that it has 42 pins. Ours uses a property instead. The vast majority of clock and power management drivers for other SoCs hard code the clock or power hierarchy and provide static sets of outputs. We put every single clock control in a separate DT node and describe their relationships. A typical cpufreq driver has intimate knowledge of the clocking controls for the whole SoC, and hardcodes the layout of the clock registers. My prototype for that just has a separate instance for each CPU cluster, and describes the performance states in the DT. And so on and so forth.
Basically, on a typical SoC, either a hardware block is identical to last generation's, or it's incompatible. Apple's blocks instead follow patterns, so we're building parameterizable bindings that can handle any configuration of those blocks with a single driver.
> the M1’s CPUs are so powerful that a software-rendered desktop is actually faster on them than on e.g. Rockchip ARM64 machines with hardware acceleration.
But to me I think this project will be indeed ready in a few years, and I will certainly be running this on my M1 as soon as it is stable and useful. It will be interesting to see how this turns out, I just wanted to comment on the next-level narcissism going on. Why most of you choose to be pessimistic and make not your problem, your problem, is beyond me.
Keep up the great work Team, Asahi!
No. I criticise this project because it adds value to a device that we should all be boycotting.
We absolutely do not want to see the proliferation of custom, locked down SoCs on the desktop platform with each one being incompatible with each other, and limiting our freedom to run what code we want on it. That is why this project is extremely short-sighted for the future of our computing freedom.
The M1 is a locked blackbox. It is designed to take away user control on both hardware and software. These are legitimate criticism that many Apple fans try to deflect. They claim that the M1 isn't a locked down machine by comparing it with the ios platform as proof. After all, iPhones and iPads have locked bootloaders that prevent you from even running any other OS, while this is not so with the M1 computers.
That's just plain denial. Just look at what has been happening to the Mac Mini:
1. The first few Intel Mac Minis allowed you some level of customisation of both the hardware (change RAM or HDD / SSD) and software (install other full featured OS).
2. Then came the Mac Minis with soldered RAM and SSD. You could no longer customise the hardware. Software was still customisable and you could still install other OSes. (Recall that Apple even offered free drivers for another OS, i.e. Windows).
3. The current generation of M1 Mini now doesn't allow you to customise both the hardware (everything is soldered) and the software. Technically you can install other OSes, but the reality is that currently only crippled versions of Linux and xBSD is available and practically the only full-featured OS available for it is macOS.
These are clear indicators of how Apple has been working slowly to lockdown the Mac platform like their ios platforms. (Right now, projects like these give Apple and M1 publicity without harming their end goals. And so they are tolerated. Want to bet that as soon as some alternate fully featured viable OS appears for the M1, the bootloader will be locked, and the next Apple SoC will cripple it again?).
The frog is still slowly boiling - https://en.wikipedia.org/wiki/Boiling_frog - to keep you in denial.
There's another reason why I call this particular project short-sighted. Remember what happened when Apple introduced the Mini with soldered RAM and SSD's? It wasn't popular and didn't sell. Apple was forced to backtrack and the next Mini didn't have soldered RAM (but the SSD was still soldered). A similar thing could have been possible with the M1 too. Apple has bet their future on the success of their ARM processors. But if people boycotted Apple Silicon desktop platform for not being as open as AMD / Intel, Apple would have been forced to compromise a bit (at least strategically for the short-term) and released more literature to make the platform seem a little more open. And we might have seen Linux and xBSD being supported on the platform by now.
That's a pretty loaded, holier than thou attitude.
Keep your politics out of my computing and open source, please.
Then ignore me and go do the politics that you want rather than unnecessarily choosing to target someone with whom you don't agree or want to meaningfully engage.
Open source started as a political movement.
Are you gonna ask me to keep the government out of your Medicare next? :)
(no, that's not an invitation to debate broader politics, it's just the first example that springs to mind)
GNU is not the sole foundation of open source.
That the open source and hacking cultures of the 70s, 80s, and 90s weren't founded on an intentional rejection of the increasing balkanization of software and technology after the 1974 determination that software was copyrightable?
I understand, at an individual level, it might seem like throwing some code up on Github with a permissive license might not seem like a political act. But the history of open source is inseparable from the politics of intellectual property, copyright, and all those things that follow, including issues like the right to repair.
As someone who has been in the "open source for the common good" camp, there seems to be a more extreme "open source as a religion" camp.
You're not off and have given me something to thing about.
FWIW Berkley wasn't immediately what I was thinking about, but I am more on the BSD vs GNU side of things.
I've got an M1 Mini laying around (Apple annoyed me enough in the past six months that I've gone away from them entirely, replacing a M1 Mini with an ODroid N2+, among other things). If it's daily drivable, I suppose I should see about getting Linux installed on it. I've not gotten around to selling it yet...
Not that I'm unsupportive of the Asahi project, it's just a fact that you're dong Apple a favor by keeping that M1 Mini around collecting dust, while every passing day it becomes increasingly irrelevant WRT offsetting new unit sales.
If I don't have to work with iOS anymore then I'll never even buy another Mac.
Hey, if you're running Linux, you're using my drivers either way ;-)
Now back to typing away at Panfrost on my M1 Linux to debug an issue on the Odroid N2 I have Ethernet connected to the M1...
I'm curious what these corner cases are; could anyone share?
It's not the fucking "maximum BAR size is tiny" problem again is it? (hello Rockchip :D)
And then if you don't do that you run into problems with apps mapping GPU memory and doing unaligned accesses (plus it's a performance problem).
Generally I don't think "normal" is necessary? In FreeBSD/aarch64 we interpret most ioremaps (all other than WC and WB) as "device": https://reviews.freebsd.org/D20789
and there doesn't seem to be a performance problem. Well, I haven't scientifically tested the performance but SuperTuxKart can do >100fps at 4K on an RX 480 :)
Where does that limitation on the M1 come from anyway?
Amazing work :-)
> Asahi means “rising sun” in Japanese, and it is also the name of an apple cultivar. 旭りんご (asahi ringo) is what we know as the McIntosh Apple, the apple variety that gave the Mac its name.
Anyone have experience with doing this for Apple hardware? Does the company come after you if you reveal some of this knowledge to the wider public?
Not asking for myself, BTW, as I have had no affiliation with Apple.
https://wiki.netbsd.org/ports/evbarm/apple/ for NetBSD install instructions.
I don't see a focus on the FreeBSD side (yet?) however.
We're also dual licensing all our bespoke drivers, so the BSDs can take code from there (particularly important for the GPU driver, as there is already a lot of shared code in that subsystem).
I'm actually thinking I'm going to rewrite the WiFi support patch for Linux from scratch, based on the OpenBSD version, just because they did a great job distilling what matters out of the original messy PoC patch that Corellium dumped earlier this year.
why not run Parallels since it now uses the native macOS hypervisor.framework, and then Linux / Haiku / FreeBSD / whatever?
i thought of that as it somewhat outsources device driver digressions.
Support can be pulled out from underneath you at any point and you're limited to the exposed hardware interfaces.
Throw on top that (at least for me) I have zero desire to maintain a macOS machine.
(Plus, we actually support the M1's vGIC which Hypervisor.framework does not yet, so VMs running on Linux should perform better than VMs running on macOS! Yes, we beat Apple at supporting some parts of the M1 already.)