This is a very macOS problem. Why does it reset every preference on updates? The one that kills me is wake-on-lan, that gets turned back on regardless. Hate!
A macOS update isn’t “whatever was previously on your OS volume, plus arbitrary patch X”; rather it’s “a new, fresh OS disk image, written to a separate APFS volume, with a fixed SHA, with update transfer-size optimized by composing said image partially from files in your current OS, but only in such a way that the volume will still hash the same in the end.”
(In many ways, macOS’s update system now reuses the logic from iOS’s IPSW firmware-update system. You can even now download macOS updates from Apple as IPSW files, and then use Apple Configurator to push them to macOS devices.)
Unlike ChromeOS/CoreOS, after the first-round SHA verification of the volume, macOS will then patch the new OS-base-image volume with certain files from your current OS-base-image volume, if 1. they’re distinct from the ones it expected to be there, and 2. they appear on a whitelist of known-safe files.
If any of this patching happens, the OS volume’s metadata tree will then be re-hashed, and the new hash will be blessed by the volume-signing utility.
(If you’ve seen a “Recovered Files” directory on your desktop after an update, and it’s contained a “foo.system_default” file, that’s a copy of the file that macOS would have composed into the OS volume if it hadn’t found your known-safe customized file to use instead. /etc/shells is a usual trigger.)
Some preference files in /System/Library are known-safe; but others aren’t. It’s a whitelist, and it’s a conservative one.
Preference files in /Library live in the user volume, not the OS volume, and so will never be wiped. This is why most of the “trivial” preferences stick around.
Note that by “known-safe” here, I don’t mean “can’t be customized to malicious ends.” This isn’t a security thing. It’s an ABI stability thing. These files are safe in that they’re known to have the same ABI format between OS versions, and so keeping a customized version of them around won’t confuse newer versions of system daemons.
It’s about the same as if Linux had a set of known-safe DKMS modules that use interfaces that the kernel intentionally doesn’t change between releases, such that the kernel module files composed from those DKMS modules could be reused by the new kernel without requiring decompilation, rather than being tossed out at every single kernel update.
The richest company in the world can't think different?
In my hypothetical, there’s a set of preferences overrides Apple have been forced by contract to guarantee they’ll validate during their QA process, and then allow the OS to retain between updates. For any other file not forced by contract, they don’t bother with that QA process (“takes too long!”), and so default-distrust the ABI-stability of the file, and so don’t let the OS retain it between updates.
I actually wrote a white paper on just this in ~2003 or so - and met with several engineers from google and they said it was impossible.
The idea being that you only carried around with you your profile, and you could just come to a dumb terminal, plug in your key, three factor auth - and the terminal would give you access to the apps and resources they had...
(I should write ((again)) a short story on this)
What reasons did they name?
There was a phone that came out (nokia??) that had a docking station and you could have a screen on it and a keyboard attachement etc...
I presented this in ~2004-ish as a white paper... (lost to hundreds of machines since, and poor data mgmt over time)
I met with a few engineers friends from google over this idea and they said they didnt think it was possible (which at the time it was not - even though I wrote about it in 2001) - and then there was that phone that came out which allowed you to have a phone docking station and use it as your primary computer.
This was at the same time whilst I was talking to my buddies at Intel about stacked procs - and they were doing 64 cores in ~2004 in test dies... and they worked.
It was in 1998 when I was at Intel that I asked "Why cant we just stack multiple CPUs on top of eachother?" and was laughed at...
I sat right next to Andy Grove, but I only spent all my time in the DRG Game lab testing games on AMD and Celeron procs to get subjective results on game perf.
Anyway... I posed a lot of things that were laughed at, which then became reality later.
I worked with a MIPS proc eng about Slot-Rack-Servers, in 1995 - this was deemed impossible... later we have HPE based systems... His name was Kent... he was one of the chief MIPS designers...
I was saying "lets make 'slots' that we can install a switch, a server, storage or whatever on the backplane..."
Yes - I am not lying - these were just things I imagined in the early-mid-late 90s....
I had a good career - but I cannot take any credit for these taking production due to I didnt ever implement any of these ideas... aside from expressing them earlier than others who were far more capable of executing.
Its just like IFTT - I literally whiteboarded IFTT for a bunch of engineers from Lockheed a few years before that existed... They have built a lot of what you interact daily with (netflix) among others...
My soul flaw - is that I think of something early, I have no ability to bring it to fruition and even though I have exceptional famous engineers as friends, I cant bring my ideas to market...
and then a few years later - things I designed hit the zeitgeist and hit market...
I have, as a consultant, made people MANY billions of dollars - and havent received anything in return.
Opinions may differ on how well it was executed in practice. I'm not sure /etc/ with its hundreds of different file formats and Ansible or Chefs as an 'API' is that much better
Was it? I got the impression that the original (Windows 3.1) registry was a Windows-internal thing—a store of Windows settings, and a set of APIs to read and modify those Windows settings, e.g. COM/OLE class registrations. (See https://devblogs.microsoft.com/oldnewthing/20080117-00/?p=23...)
But then, third-party ISVs found the registry, and exploited it to store their own settings. And Microsoft being Microsoft, they accepted that unilateral design change and continued on with it.
With the registry, each vendor puts its stuff under a path like HKEY_LOCAL_MACHINE\Software\VendorName, even Microsoft.
The registry was the next step. What would be great is an online serve for such - a machine gets booted and it asks for your config unam UUID etc - and then just slurps your snapshot down.
Malware can't do that, because there's no way for any executable that runs in the regular OS—and isn't signed by Apple—to get anything to automatically happen over in the Recovery OS.
(That's not to say you can't have persistent malware in macOS; just that it can't persist itself into the OS boot volume. It has to persist into the user volume—which is great, because that means the OS can, on boot, mount the user volume noexec and scan it for malware while running in a known-good base state. That's even without needing to boot into the recovery OS.)
So, other than OS updates, basically the only reason these files get modified is when people modify them manually. And the only people doing that are 1. people developing kernel extensions (or their friends, the Hackintosh community); and 2. enterprises burning low-level configuration changes into OS images for image-based deployment.
And both of those cases involve modifying things that are effectively "underneath the OS API abstraction", and therefore don't have the same ABI guarantees that the OS APIs themselves do. Thus the whitelist.
Yeah - fuck apple - and fuck Jonny Ive - they claim to be the masers of all aspects of designs, but really they're they masters of /r/assholedesign
Their hardware is in the top 70th % - but their UX is IMO <25%
(just look at how many clicks you need to do certain tasks, such as bluetooth. They have a slide-up menu to toggle BT and WIFI - but you have to go to desktop->settings->bluetooth->(toggle it on off/refresh)->find the device-> "cant connect" - Toggle BT on both phone and device -> attempt to connect...
But you cant up-swipe hit BT and have it show you the FN menu on screen.
You cant backup and manage all your prefs via icloud - such that you can apply profiles, save profiles, etc from your device to your linked cloud account and say "I always want my privacy to be thus, these are the networks I trust.
Their photos library mgmt is absolute garbage. You have no photo details available to you.
Their albums are garbage.
There are so many interactions that require like 5x more clicks than they should.
Their screens suck.
Their device accessory ecosystem sucks and punishes you for profit with impunity.
They exploit chinese slave labor.
They try to charge you for flaws in the HW provided (recall the balloon battery problem?
I had a macbook pro CATCH FIRE while I was asleep in bed. They said they "recognize that was a safety problem, but because our engineers (after two months of having the machine) determined that at one time the liquid sensor was set off, we cannot replace your machine EVEN THOUGH IT WAS UNDER RECALL FOR THE CLASS OF PROS THAT WERE RECALLED FOR CATCHING FIRE.
FUCK APPLE.
Things are always a trade-off. There are big positives to keeping the ecosystem on a consistent, new and secure version of iOS. Similar positives apply to MacOS.
I have literally had every single apple product since the Apple ][e -- aside from their server systems.
I have had every iPhone (even had pre-launch phones from them) since launch up until I stopped at the 6s+ which is the only phone I will ever own from them.
I have a history with apple that is not shallow and I have had ~35 years of judgement against them.
I recall getting into a talk with a fellow engineer at Lockheed where I said "Trust me, Apple will switch to intel procs - and this apple boi almost had a heart attack saying that would NEVER happen...
https://en.wikipedia.org/wiki/Mac_transition_to_Intel_proce ssors
Yeah that happened ~2 years before it actually occured...
(I used to work for intel and have stories about trend prediction there as well...)
So yeah, I'm solid in my position of what I know to have happened...
Windows gets into this state fairly quickly, too, but the failures are less dramatic so we just live with it.
At least with Debian, it only happens when I deliberately change stuff! Though the `xorg-server-video-intel` driver breaks features of post-2007 Intel cards, and it's installed by default, so… perhaps it isn't the perfect OS either.
- If macOS is the only OS that you use regularly that gets in a fucked up state, then either you're not using Windows or it's gotten a lot better in the last few years. :) (I mean, it undeniably has gotten better, but I have Windows-using acquaintances who still kvetch about this issue pretty regularly.)
- I've been using Macs since 1999 and I don't think I've had to reset the PRAM to fix a problem since my Titanium MacBook Pro circa 2007. I've had to nuke other settings on occasion, but still generally have markedly fewer problems than I did in my Windows-using days. (Which were more recent than 1999, but still not that recent, so back to the "I'm sure it's better now" disclaimer.)
But, Apple definitely needs to have a better mechanism for managing config file updates than "yeah, we've put a few old config files in subfolders of this 'RecoveredFiles' folder, maybe they're useful, maybe they're not, good luck."
Though the difference is, if something gets royally screwed up on MacOS, at least there's a sane enough way to reset it. On Windows, it often entails a reformat.
And let's not get started on the Windows Updates. They too have the whole "reset settings" problem (e.g. "don't f&!king restart my computer randomly", which gets reset every somewhat major update).
As for the rest, my suspicion is that the newer "mobile generation" OSes both tend to get messed up much less often this way and tend to have much less recourse to the user when they do. That's certainly true for iOS/iPadOS and I suspect it's true for ChromeOS; I suspect Android is probably somewhere between iOS and Linux on that scale.
Red Hat CoreOS (and other Linux systems managed with OSTree, such as Fedora Silverblue) now do something similar by merging the current /etc directory with the "upstream" version.
I haven’t looked into it specifically, but I’m guessing that like many other basically-static system policy property-lists in macOS, the whitelist lives embedded in a Kernel Extension (kext)—probably the kext responsible for update validation. So you’d have to write your own kext to either replace that extension wholesale, or—hopefully—“inject” over just that element of it.
Of course, to run your own Kernel Extensions these days, you have to run your Mac in bputil --enable-kexts mode, which is “reduced security” from Apple’s perspective (reduced compared to the security model of iOS, they mean.)
On the other hand, the Hackintosh crowd have long agreed that it’s better to just “inject” data into the kernel/OS by either memory-patching the kernel during early boot (Clover), or patching the kernel as it’s being composed into a loadable image (OpenCore), rather than actually modifying anything in /System. (This was before modifying /System was even hard; they just didn’t like making a mess!)
I have a strong feeling that that same community that loves their little quality-of-life kext tweaks so very much, is pretty likely to come out with a way to use OpenCore as the boot-loader on real Apple-Silicon Macs, and thus to overlay arbitrary kexts into macOS without actually reducing macOS’s (OS-stage) security policy at all. (Provided you’re okay with the reduction in security of using a third-party bootloader, that is!)
The file in question is configuration and should not be part of the update image. (Better yet, Apple should surface this functionality somewhere reasonable, like the System Preferences.)
There was an issue with iOS a few years ago that caused higher than normal battery drain when I was managing a department that supported a lot of iOS devices. A lot of users thought they needed a new battery. The battery drain issue was resolved for 100% of our users by doing an "Erase all data" and NOT restoring the backup. This process will still fix an amazing number of weird iOS issues in 2021, especially issues with the device slowing down, getting hot, or draining battery. It works because you end up with default config files that were designed for that version of the OS. If you restore the backup, often it will "restore" the problem you were trying to fix because the iOS restore process puts some OS config files back.
It will be interesting to see if Macos becomes less susceptible to win-rot type issues because of this config file reset behavior, as annoying as it may seem.
After awhile they would all become unusable, and I would reinstall everything back to default and try again. Great for leaning, lousy for day to day work.
Resetting things back to a known state can make life lot easier.
In 10 years I can’t recall needing to reset a Mac.