macOS extensions are moving away from the kernel
eclecticlight.co
eclecticlight.co
So there is one year to try out what might still be missing and give respective feedback.
The long term roadmap is that macOS will eventually be much more microkernel like.
The monolithic vs microkernel debate was pretty clueless at times. Core Linux is smaller than dito hurd when configured with similar functionality.
OSX mach has always been a hybrid kernel. In a sense Linux is too but you can also configure it as a mostly-monolithic thing, and most seem to prefer that.
Likewise, eBPF allows userspace programs to perform some work inside kernel space to reduce syscalls.
I wonder if io-uring won't be a starting point for a "microkernelization" of Linux. It seems really effective as a message-passing interface.
https://source.android.com/devices/architecture/hal-types
The Switch (yes the games console) uses a microkernel.
QNX an OS for human critical scenarios is a microkernel.
Finally, when arguing Linux vs Minix/Hurd, think of the irony of running Linux on top of a cloud hypervisor, with an endless load of Kubernetes or Docker loads.
Hmm, Never thought about a stated OS focus like that. Off to look it up. Good mention
The fact that they're no longer going to allow loading of 3rd party kernel modules (a pure political decision) does not a make a microkernel (which is an architectural style).
The kernel is still going to be big and monolithic (it contains the entire BSD kernel in the same address space!), and they're still going to be loading Apple-authorized kernel modules (500MiB of them, according to the article) in the same address space.
I rather be happy for the improvements in the long term roadmap and overall outcome for the eco-system.
If you just consider disallowing 3rd party modules to be a "microkernel" all while the kernel itself still loads most if not all (approved) drivers such as PCI bus, storage devices or filesystems in the same address space, then _all_ current major operating systems are "microkernels".
What you call a kernel like that?
Locked?
"A kernel that only allows user-space third party drivers"?
That this style doesn't have a nifty name isn't a good reason to weaken the established definition of microkernel just so it does.
So, then it becomes a balancing act how many are allowed the priviledge to be part of the main act.
Well, that is kinda the point of a microkernel (or at least one of the points).
Sure, you're not going to have a kernel whatsoever without say an CPU/MMU driver in the same address space. Maybe a driver for the timer.
But when you have graphics, sound, storage, USB controllers, filesystems, networking and most of a BSD kernel (providing all the syscalls) in the same address space, now that is not a microkernel by any definition of the word.
XNU is called an "hybrid" kernel in part because in the original design the BSD parts implementing most of the actual system could have been spun off as a separate process (similar to Windows NT's original design), which would have been an actual microkernel, but in practice they were just stuck together in the same process, making it a monolithic kernel in all but name (just like in Windows).
> So, then it becomes a balancing act how many are allowed the priviledge to be part of the main act.
Yes.
Which is what pjmlp implies, once they are out, isn't it microkernel?
What you call a kernel like that?
This isn't pedantic, nor is it ideology. If you're acquiescing, do it graciously and please refrain from name calling.
From all purposes from a third party driver writer persona, the only path forward are user space drivers.
What you call a kernel like that?
This is basically calling me silly, just as you were calling the previous person ideological and a pedant. If you cut out this commentary that would be great.
macOS does not use “the” BSD kernel (it’s not even obvious what your assertion even means since there are many members of the BSD family, each with their own kernel - none of which macOS uses).
Using a USB network adapter (requires for Ethernet on a MacBook) would bring down my system every time the firewall was running. And the stacktraces all included their calls.
I’m sure Apple, like me, got tired of getting reports about a few known bad actors.
From what I remember reading from some macOS Application Firewall makers, no. (And that's ofcourse, intentional.)
While it is true that poorly coded Kernel extensions can make an OS unstable, this is just another example of Apple taking away more control from its users and further crippling macOS to make it more like ios.
I still think that it’s a worse in-practice system than kexts but they are responding to feedback at least.
You can still load kernel extensions in Big Sur by disabling SIP, though.
That, however, isn't easy to do, considering Apple has a rather impressive track record of creating products people enjoy and buy.
And to jump two replies ahead: them trying to make it "easy" and "save" doesn't imply their users are idiots. Quite a few developers and scientists use Macs, and the lawyers aren't exactly dumb either. Spending hours trying to somehow get both sound and bluetooth to work at the same time, a favourite pastime on Linux, means lifetime wasted for something that shouldn't need doing. And considering nobody reads the source, there is no reason to believe some "expert" has some ability to avoid installing that one extension that soon starts encrypting their files. At least they tend to have better backups...
I don’t like the way this is going.
Apple can. I can’t. Making APFS snapshots requires special entitlements, which they’re not going to give me.
> It also use checksums for filesystem integrity.
It doesn’t check integrity of the actual data though. But I care about my data.
Or do you want to create an app that performs a snapshot via some API.
That’s not what I want. I want to have a say in snapshot retention, not have some tool make the decision for me.
Specifically, I need precise control over snapshot retention so I can maintain several off-site replicas of my backup.
What makes you think that? Carbon Copy Cloner does this, and is made by a small independent developer.
https://bombich.com/kb/ccc6/leveraging-snapshots-on-apfs-vol...
Source: https://bombich.com/blog/2018/03/30/building-better-backups-...
They’re one of two apps in the world who have been granted the entitlement as far as I know. The other one is Time Machine.
It isn't. They are making it (very) less likely for average simpleton users like myself to expose or fuck up their system.
And for the more elite hackers such as yourself who absolutely want to load third party kernel extensions, they require you to disable SIP. The knowledge and work it takes to do that is a very nice "I hereby declare I know what I'm doing and I alone bear the responsibility if things go wrong" entry barrier.
In any case, extensions run in the same address space as the rest of the kernel making any such "rings" pointless.
But then, I haven't delved into how macOS in M1 precisely works.
Yes, I know, but I'm referring to hardware virtualization, essentially making "ring -1" "ring 0" and "ring 0" "ring 1".
Besides that, hyperv seems make use of it to achieve paravirtulization?
macOS on the M1 uses this to prohibit most kernel code from touching pagetables. You have to do that from within the guarded sublevel, which kernel extensions can only call into.
This is more or less the original intent of x86 rings: to separate an OS kernel out into parts with different permissions and levels of isolation.
Why did you use the word Blessed?
I'm curious what made your brain choose that word instead of something neutral.
Reminds me of "Walled garden" marketing speak instead of "Walled Prison".
Edit- is there some sensitive nerve I hit?
I'm curious what made your brain choose that word instead of something neutral.
people use metaphors with a lot of conotation (walled prison) so if there is a more neutral term, i was curious how to state it in either non-positive or non-negative way
Probably the nerve that's tired of people going out of their way to be offended over terms that never were offensive when used in context such as this subject matter.
Context does matter.
It's a fairly common idiom when talking about platform APIs. Some APIs are "blessed" in that they are the APIs that they platform owners would like you to use and others they would like you to avoid. "Blessed" isn't wholly positive as it implies the platform plays favorites, and frequently the new blessed apis don't support all the old functionality.
Here is patio11 using the idiom 7 years ago:
https://news.ycombinator.com/item?id=7658396
It's not as nefarious as you are implying - I don't have an agenda, I've just been on HN for too long.
I would have trusted literally anyone else to do a good job at this. Apple's in the business of brokering power though, I see no reason why they'd be motivated to make those APIs as rich as kexts.
Because of that, nobody would be stupid enough to make those APIs (in the wide sense, as in “what kexts can do, not what they are supposed to do) as rich as kexts.
It's really sad that we have come to this. The platform became more and more closed off in the name of security and no serious law or regulation is preventing that.
And if at least it was bug free it would be worth it, maybe. But my kernel extensions are all for circumventing bugs or stuff that does not work properly, e.g. Karabiner for remapping the ctrl character of my keyboard to command, necessarily since I work with a lot of OSes at a time.
> Changed the virtual keyboard and mouse implementation to DriverKit from deprecated kernel extension.
https://karabiner-elements.pqrs.org/docs/releasenotes/#karab...
On a 2019 mac pro, catalina, btw, looking at Settings > Keyboard > Modifier Keys...
This article [1] suggests that it was the case at least in Catalina given the screenshots - but I don't have it easily available to test.
- XNU was born from a fork of the Mach microkernel
- When the microkernel hype simmered down due to them being way too slow for '90s machines, NeXT bolted on lots of chunks taken from BSD to speed up XNU
- After 20+ years, computers are fast enough to run true microkernels, so Apple is bringing them back again
So it's finally time for microkernels to shine again?
QNX, Symbian, Nintendo Switch, type 1 hypervisors,....
So to your point: all NeXT was essentially doing was turning Mach 3 back into how Mach 2 worked lol
Thus XNU is a novel implementation of Mach 2: an up to date OSFMK kernel and all its IPC, plus all the BSD parts that used to live in it
This does mean that some older gear which requires bespoke drivers and is no longer maintained will never work with new MacOS - one example that comes to mind is Allen and Heath’s Xone DB2 DJ mixer. Quite a lot of newer gear is USB class compliant, which means it doesn’t need specific drivers and so should be future proof.
The new Bitwig 4 even dropped support for 10.13 which was a bit unexpected as it supports Windows 7.
Old gear getting dropped because of driver issues is really annoying, but I have come to terms with it. All music produces and studios deal with it. Some mixing rooms are on 20 year old hardware and that’s just how it is.
Yeah, certainly on macOS. Not much you can do about it sadly!
We have multiple Apple hardware based music systems 75% of which are deliberately still, not stuck, on 10.13.6.. We also have a few M1s.
After decades of continual forced disconnects/'upgrades' of software/hardware compatibility we drew a line at 10.13.6 and have been very happy and have not missed a single so-called new feature, both from Mac OS or any DAW we use.
We are, however, ready to move to Apple silicon, based on our early assessments.
Do you have a blog or anything with write ups about your equipment? Would be very interested in hearing about your silicon upgrades once you get there.
No blog. We don't even have a website as work has only ever arrived by word of mouth. I am not enamoured by studio blogs, generally. There's too much noise out there without my clumsy typehand adding to it. I keep abreast of technology and audio by furiously skimming.
The change over is not set in stone. I have concerns about the privacy claims made by Apple for the sake of my clients, but so far we've had little issue with our Air or Mini.
Apple's been losing ground with creative-types across all industries and areas since their post-2000s peak: the major attitude-adjustment came with the reputational harm that came from the launch-and-downfall of the 2013 Mac Pro.
As a wry zinger, I find it ironic that creative-types cannot express their creativity, and especially their individually by way of system customization - nor explore opportunities for out-of-the-box solutions to their computing problems due to Apple's uniform hardware design and their locking-down of the platform. Remember when people used to apply custom skins to MacOS?
Secure defaults is fine - I actively encourage it - but Apple is increasingly treating their desktop/laptop computing platform as though they own the hardware that's legally yours. It's my laptop and I want to root my own OS, why won't they let me?
Anecdote: I had an iPad pro I put in a drawer for a while. I had it synced to an apple account from my old job (used my corporate email for it).
When I took it out after I moved jobs, it became permanently locked with no recourse because I didn't have access to the email anymore.
And there was no way to factrory reset it, so it became as useful as a coaster. Really pushed me away from mobile apple devices...
For anyone else reading this - take heed. It's better to ensure you can recover before you need to. Apple provides multiple ways to ensure you have access to an important Apple ID - with or without access to email.
You can generate a recovery key that will work whether or not you have access to the email or phones associated with the Apple ID: https://support.apple.com/en-us/HT208072
When you now create an Apple ID it nags the crap out of you to print and file it. If you created your Apple ID before they offered the recovery keys, now's a great time to go generate it and stash it away in a safe space!
You can also set up backup/secondary emails and phone numbers in your Apple ID - not a bad idea either.
It's easy to overlook this kind of stuff when setting up ID's, and not all vendors justify this kind of diligence when you create a new account. But with Apple because of device lock (which is a useful feature for theft deterrence or just petty revenge that a damn thief will get no benefit from my stolen stuff), any data or purchases you may have associated with that account it's well worth ensuring you will be able to recover your ID before you are in a situation where you need it :)
I'm just very used to holding a paper clip in a hole for a minute and getting to use a device fresh, I guess!
I don’t think I know anyone IRL who is worried about the Mac becoming locked down - iOS sure, but you can still basically do what you want on a Mac for most people.
The reality is that for most people the Mac is a relatively hassle free experience. You don’t need to worry about whether it has certain components that will affect audio performance etc. - my understanding is you still need to be careful with chipset etc. on PCs.
Currently, it's installed as a kernel extension, but I'm not sure if it will be possible for it to move to the new framework.
I know they are having a dialog with Apple about what's still lacking for them to do it the new "right way" but I think for now you sill have to basically do what Jason outlines.
I don't know what happened there... Apple used to make wonderful keyboards, but it seems like every model they make now just plain sucks.
They already have.
[1] https://github.com/pqrs-org/Karabiner-Elements/issues/2760
[EDIT: see sibling comment - developer confirmed the above.]
Are there any users of these extensions at this point?
It does seem a bit weird that installing these extensions supposedly doesn't require any user consent, even though stuff like screen recording or folder access does.
I have VMware fusion running (latest version), so maybe they've finished the transition?
% systemextensionsctl list
1 extension(s)
--- com.apple.system_extension.network_extension
enabled active teamID bundleID (version) name [state]
* * MLZF7K7B5R at.obdev.littlesnitch.networkextension (5.2.2/6209) Little Snitch Network Extension [activated enabled]Seems they just want to lock it down to squash developers making kernel level features for mac...
I'm sure someone will find a back door to allow extensions somehow, but we can't keep fighting forever :(
The problem is that most extensions are designed to work on all web sites, so you have to choose between security and convenience. https://yvanced.com/
Most users pick the latter and trust the former.