Apple XNU: Clutch Scheduler | Hacker News Reader

Apple XNU: Clutch Scheduler

github.com

181 pointstosh48 comments

interesting. id love an eclecticlight breakdown of this. they're one of the few if only that write anything worth reading on apple hardware, i once found a QOS/scheduler insight through those guys when I couldn't get my c/cpp project pinned to the cores I wanted on m-series. https://eclecticlight.co/m1-macs/

He doesn’t read code. Convert it to some log messages and he might explore it.

Please take anything this fella writes with several grains of salt.

Please could you elaborate?

I mean that goes without saying on anything I read on the damn internet. With that said, I'm not seeing a comparative informational source anywhere that even takes the time of day. Feel free to bombard me with new resources.

> The XNU kernel runs on a variety of platforms

This is fascinating, would love to know where it’s used! (Besides macOS)

Perhaps they mean ISAs

Well x86 at one point, arm both the 32 and 64 bit versions. I think they had RISCV support in their source tree at one point but not really at a commercial level. It does cover a lot different levels of hardware though

PPC32/64 of course, and for a long time Darwin still contained remnants of its predecessor's support for SPARC, PA-RISC, and m68k.

Is mc68k or PPC still in there anywhere?

I'm sure there's vestiges of them somewhere, but the underlying support (the architecture specific parts of the mach portion of the kernel) is gone for those archs.

I wouldn't be surprised if they keep a minimal Power base maintained behind closed doors. It's how they managed to jump ship to intel so quickly, they never stopped maintaining NeXTSTEPs x86 port

I would honestly be shocked if they were.

They've been making quite a few changes to the virtual memory code over the past decade, and keeping those vestigial arch's around is a pretty big maintenance burden. It'd probably be less work to just add the arch as if it were new when it's needed at this point since the kernel itself is pretty portable.

I seriously doubt it.

Apple's ARM implementation is in a really good place right now. It would take something extremely compelling to get them to consider any other architecture for an application processor, especially considering that it'd mean giving up some degree of control.

Power is probably not where Apple would choose to go unless something really unusual happened. It's essentially just IBM's pet architecture at this point.

When I mean minimal arm base, I basically mean making sure XNU at least compiles on PPC64. I don't doubt they'd likely never have a use for it, but they maintained intel support behind closed doors when macos forked from NeXTstep and nobody thought they'd need that

Which Apple products run arm32 XNU? Their first Apple Silicon CPUs were already arm64.

Well there were still the historical arm32 chips in their iOS devices, but until recently the watches were a cursed arm64_32 (or something like that) which is arm64 with 32 bit pointers iirc.

I should have just soureced this, They had PowerPC not RISCV in there source tree that was the X factor one. The Arm32 bit variant is closed sourced (leaked before) but was supported until IOS 11. XNU is really old almost 30 years! And before XNU there was the MACH kernel and the larger BSD tree it was built on which is an argument that it probably had a initial MIPs release too but I couldn't source the truth on that.

[1] https://en.wikipedia.org/wiki/XNU

Does Apple use macOS in servers in its datacentres? Or are they all Linux?

Surely at a minimum they need macOS for CI.

Apple does have one advantage here-they can legally grant themselves permission to run macOS internally on non-Apple hardware, and I don’t believe doing so legally obliges them to extend the same allowance to their customers.

But that might give them a reason to keep x86_64 alive for internal use, since that platform (still) gives you more options for server-class hardware than ARM does

They do run Apple Silicon in data centers, so perhaps another custom version of Darwin + their system frameworks. It is hard to tell without some leaks :)

For Private Cloud Compute: “a new operating system: a hardened subset of the foundations of iOS and macOS tailored to support Large Language Model (LLM) inference workloads while presenting an extremely narrow attack surface.” https://security.apple.com/blog/private-cloud-compute/

I wonder if there is any chance we might see another Xserve?

If they’ve got Apple Silicon servers in their own data centres…

They use Ubuntu on x86-64 servers, at least for iCloud. Backends for iCloud, Photos and Backups etc. are written in Java.

Any sources or more information on that?

For the Java bit at least, this aligns with job descriptions I’ve seen and recruiter outreach I’ve received (long time ago though, maybe 5 years).

NeXT added a Java variant to WebObjects and it was for several years the main server side infrastructure, after being acquired by Apple.

Nowadays you can usually still find Java and JVM languages like Clojure (Apple Maps), on Apple's job ads.

How much of it is still Java based, no idea.

I imagine XCode Cloud has nothing to with it for example.

They moved their password monitoring service from Java to Swift.

https://news.ycombinator.com/item?id=44172166

https://www.swift.org/blog/swift-at-apple-migrating-the-pass...

Unfortunately I am the source in this case. It is from having worked on them personally. :)

IIRC, Apple uses 'platform' to refer to an SoC integration. For example, M1, M2 and etc. are separate platforms. M5 in Vision Pro is a separate platform than M5 in MacBook Pro. I believe Apple's XNU does somewhat still support non-Apple Silicon as well though.

Yeah they're was that whole x86 thing thru did for quite a while.

Twice, on the basis that NEXT used the same kernel and that ran on 68k and Intel when Apple bought them and later ported it for Power PC. When Steve Jobs went back to Apple, for a long time he ran NEXT on a Thinkpad.

NeXTSTEP also ran on SPARC iirc

OpenSTEP actually.

OpenSTEP had SPARC support, yes, but NeXTSTEPs last release had support for m68k, x86, and SPARC. 3.3 had support for PA-RISC

I believe it means Apple's other hardware platforms (phones, tablets, smart TVs, VR headsets, smartwatches)

It's used in iOS as well. iOS runs in some unexpected places, like for example Studio Display. Also, the Apple Lightning Digital AV Adapter runs Darwin (because RTKit didn't exist yet).

And touchbars too, strangely enough.

For Intel platforms, the Touch Bar is driven by the trusted coprocessor (T1/T2), but that itself runs bridgeOS which indeed is Darwin/watchOS-based. With Apple Silicon I don't know if bridgeOS is still used; the SEP runs an L4.

that lightning AV adapter is crazy, iirc it creates an ethernet connection to airplay the display to the device

All of Apple's platforms.

Darwin had bunch of schedulers except this one: dualq, multiq, etc

In fact here's the one used in Sonoma: sysctl kern.sched -> edge

which seems to be an extension over "clutch":

https://github.com/apple-oss-distributions/xnu/blob/main/osf...

"… Single-cluster, symmetric (SMP) systems can run with just the Clutch policy, but multi-cluster, asymmetric (AMP) systems must further enable the Edge policy extension to Clutch in order to manage scheduling across the multiple CPU clusters. …"

Edge is also described in the linked document.

Does this contribute to macOS's suitability for DAW applications or is that more the baked in low-latency audio drivers?

CoreAudio probably deserves most of the credit, there. Similar ASIO-style solutions like JACK, DirectSound and now Pipewire hit the sub-30ms mark without any big scheduler tweaks.

IOKit was designed to support CoreAudio from the start, cc gvdl.

CoreAudio was developed alongside xnu / IOKit for Mac OS X, so it’s kind of all of it. Apple had the opportunity to start fresh with a built-in super low-latency audio subsystem at the turn of the century, and they took it.

Audio actually runs on a dedicated realtime thread. This used to be scheduled differently, but nowadays it might be implemented by the FIXPRI bucket described in this document.

Read through much of this. Definitely started feeling like “a picture might be worth 1,000+ words”.

Reply on news.ycombinator.com