macOS updates for Apple Silicon Macs are larger than reported
eclecticlight.co
eclecticlight.co
Link from the same publication on this exact topic.
https://eclecticlight.co/2020/04/09/where-did-all-that-free-...
"free" space being contextualised isn't entirely nice. better to be clear how its consumed, than say its free.
if you are an avg mac consumer running out of space constantly and hitting that issue- you ask support/genius and they tell you to buy a bigger capacity more expensive mac or icloud storage as one of their "solutions".
(a) An issue that the vast majority of the users wont ever see.
(b) Those that would see it, would only see it after they already bought a Mac (with no storage upgrade option available to them to buy from Apple for their current Mac).
(c) So Apple's is hoping that a handful of those would do what? Buy a new Mac immediately? Totally unlikely. Get their next Macs 2-5 years down the line with more disk space?
(d) And all that combined with Apple already doing several things for the macOS to need less disk space, like the "sent online, storage restored, downloadable on demand" files feature introduced in the last OS, or a couple of rounds of prunning of the OS install size.
In general this is the kind of conspiracy theory that totally misunderstands business tactics, their feasibility, and their margin for profit.
I grow weary of the internet meme that everything is some trick or scam, often including things that aren’t.
it's not as innocent as you imply it is.
edit: don't forget apple solders their storage in nowadays, so buying anxiety multiplies as you can't just fix it later.
I would guess that for more than 99% of users just saying "free" about swap/temp is the best option and it would just be confusing to say less disk is free or show multiple values. And if you really want to know you can just select "Get info" on the hard drive and see how much purgeable space you have.
Having really good background processes that makes sure the space is free when you need it and easy to understand error messages when it can't purge is of course great for everyone.
- free space is being used as swap until needed, then transparently released to be used
- "free" uniformly means being used this way, unlike "not free" which means it will not be released transparently when space is needed, so "free" is not being contextualized
- since the space is free but reported as "not free" by df, df has a bug
(edit: formatting)
If you're going to implement something like this then ffs do it right. I don't know the "easy" way to fix this, but in any case if something is supposed to be transparently free and available then standard tools using standard calls should see it as free and available, with specialized user space tools that can look behind the screens when needed.
When disk, RAM, and swap are all full, does write() sometimes cause the OOM killer to kill other processes?
> then transparently released to be used
is doing most of the heavy lifting here, because in practice it often is NOT transparently released.
To give a concrete example, imagine you have 500GB free, which is enough for you to unpack and analyze a single simulation output at a time. You unpack the first one, do your analysis, and then rm that folder and its archive to make space to pull down the next batch of results.
However, you then will find that for some reason, despite Finder saying that it has plenty of space, that trying to unpack another 500GB result will fail with an "out of space" error, unless you wait several hours/days for unnamed "background processes" to garbage collect that drive space. What that looks like in practice is that Finder will start to report superlinear space usage (e.g. for every 10GB you unpack, >>10GB of usage will be reported) until the operation fails.
Thus, I find that df actually is much better about predicting the actual expected behavior of the system than Finder itself. If there were a way to manually trigger an immediate cleanup via the command line, that would also be fine. As it is, I just work off of external drives w/o APFS, which avoids the issue entirely.
That's a common assumption but it's also a lie: because you do have to care about wearing your media, reducing its lifespan. Abusing storage with heavy doses of r/w activity (which is what swapping does) is not good. Yes, the issue is not as bad in modern "disks" as in rust-spinners, and can often look like a matter of efficiency (because of how memory is physically "flipped", on modern drives it's more efficient to write a ton of data rather than a few bytes), but in reality you are wearing your disk more than you would if macOS kept its grubby hands to itself.
Of course this is not a downside for the people who will sell you replacement disks, or rather replacement machines (since drives are now soldered), who coincidentally happen to be the people who develop the disk-wearing OS.
Unlike SSDs, hard drives never bothered to track IOP counts and accumulated read-write volume.
The downside is that the output of "df" can be confusing at first if you're not used to it!
(Maybe this feature is common on all OS's/filesystems now days, but I certainly could have used it back in my Linux-admin days. It was quite a revelation when I discovered it!)
So Time Machine is pretty space-efficient, but it will keep growing over time until it fills the entire backup drive since there is no limit on how old the backups it keeps can be[1]. I guess when it starts filling up I will just manually delete some old backups to free up space if I need it. Apparently, another alternative is to limit the size (set quota) of Time Machine's backup volume in Disk Utility.
[1] Personally I wish this could be configured. I don't really care much about keeping old files, I just want to make sure I have a recent backup just in case my MacBook gets lost or stolen or it's SSD fails...
This is also when it personalizes its boot signature (ticket) for your specific CPU via the TSS server.
This is done by transmitting your ECID (a hardware serial number for your CPU) to Apple. This is done over unencrypted plaintext port 80, allowing anyone who is passively monitoring all internet backbone traffic to associate that (and every other!) serial number with the client IP (and thus city level location). It also permits Apple to associate every ECID with a client IP and city level geolocation (and timestamp).
I have reported this behavior to Apple on multiple occasions and yet the plaintext transmission continues. Apple does not permit apps in the App Store that behave like this, but Apple's own insecure boot loader updater is exempt from this security policy apparently.
https://sneak.berlin/20220409/apple-is-still-tracking-you-wi...
Before that though I used to only install command line tools.
At some point though, I forgot the incantation for it and realized I could just use Nix for git as I was using it for other things already.
Unfortunately, as the git website [1] states, these are fairly out of date (currently 2.33 from 2021-08-30). To get something recent, you have to build them yourself, which requires...devtools.
Seems bizarre, to me, that they don't have them built automatically.
The "absurd payload size" is the effect of having at least four different SDKs (macOS, watchOS, i[Pad]OS, tvOS) bundled, including a simulator for three of them.
Xcode 15 is a ~3.5 gigs download, and includes only macOS SDKs, with all of the other (including the visionOS SDK + sim) ones being downloaded on-demand.
- using App Store for Xcode was historically very painful and ~nobody who uses Xcode daily downloads it from MAS
- it never worked ~that well~ for Xcode anyway - apart from the big download sizes, the problem with Xcode and the bundled SDKs is that they’re all approximately trillion little text files, and it’s not unheard of to wait longer for codesign (technically unxip) verification than for the download itself, which negates the user-facing benefits of delta upgrades for many users.
I've been doing this for as long as it's been available there as a developer at Apple, in my own startup, and in my current role as well. Never had a problem with it.
I never thought I'd appreciate Windows Update but on there this would be less than 100MB and take 5 minutes to install...
The requirement to reboot is supposedly fixed in Sonoma, with discrete ‘cryptexes’.
I've never experienced anything similar with any of my Windows computers, nor Linux ones, nor ChromeOS ones. MacOS stands alone as having by far the shittiest and slowest update system imaginable. Even Android during its "recompile every installed app after an OTA" phase didn't take as long as MacOS does, and that's with massively slower CPU & storage!
And maybe they will give us a GUI framework apart from HWND that will at-least last 5 years. Or integrate web standards right into the OS.
That’s certainly the case with Go/Rust with respect to C/C++.
https://everymac.com/mac-answers/snow-leopard-mac-os-x-faq/m...
Software-wise 64 bit userland processes were supported since Tiger (CLI and Cocoa only, Carbon UI stayed 32 bit) before the kernel itself moved to 64 bits in Snow Leopard (although IIRC not the default except on Xserve and Mac Pro, the two that could have loads of RAM. You could tell it to boot the 64 bit kernel through nvram boot args or something though)
A few Macs had a 64 bit CPU but a 32 bit only bootloader/firmware. That made them disqualified for one of the major OS version updates when it required a full 64 bit boot chain.
I am not sure how much of that really applies here, this seems like a simple arithmetic bug.
Yes, the unwashed masses were just getting past the nerd factor of the Internet, but it wasn’t that bad for software distribution.
The experience of using a mobile phone, on the other hand, was brutal. If you wanted to get a new feature enabled on your device, you’d only be able to take it to a retail outlet of your phone provider.
My specific examples were (1) getting a firmware update on a Nokia 6188 to enable access to the 800 MHz band of the provider’s then-restructured CDMA network; and (2) same for a CDMA Blackberry to enable EVDO data.
What? The 68k to PPC transition was from 1994 to 1998 (versions 7.1.2 to 8.1). The NeXTSTEP-derived Mac OS X (which uses .app bundles rather than resource forks) was released in 2001; though there were betas and developer releases as early as 1997, the OS transition is still distinct from the 68k to PPC transition and there were quite a few PPC-only releases of the pre-OS X Mac OS.
There certainly were during the PPC to Intel transition. This happened in 2005 with "Tiger", and at that time OS X had already been receiving updates over the Internet since quite some time.
> …but the second 1.1 GB always has to be downloaded direct from Apple’s software update servers and can’t be served from the cache, making it slower to download.
Why can’t it be cached? Because it’s unique per device. Only thing that makes sense.
There’s no reason to think the ARM package is supplemental binaries to overlay on top of an Intel package in the way your original comment stated.
All the article says is Intel gets 1 500mb package, ARM gets two packages, a 700mb and a 1.1GB. It doesn’t opine on the contents. All we know is that Intel gets 1 cacheable package and ARM gets 2, 1 cacheable and one not, and that the Intel one happens to be roughly the same size - but not exactly - as one of the ARM ones.
> It's not like you can argue that they can't pull off a RISC transition
Apple could also possibly send a spaceship to the moon if they really wanted to. Not sure there are any practical reasons to do that.
They’ve been working on ARM chips for almost a decade to get to where they are now. Throwing that away for some ill defined advantages seems not that rational.
I think Apple has a perpetual license to ARM tech because they put up a bunch of investment money after ARM spun off from Acorn.
None of us really know the specifics of Apple's deal, though. It's a perpetual license and they are in-bed with the firm that majority-owns ARM. It seems like a good deal, but obviously it's not good enough; Apple doesn't use any ARM-provided core designs and even adds their own ISA extensions when-needed. They themselves are using ARM like it is RISC-V, and they're a big enough stakeholder that they can get away with it.
> They’ve been working on ARM chips for almost a decade
https://appleinsider.com/articles/21/09/03/apple-investigati...
They've been exploring RISC-V internally for a couple years now, too.
I think it is either extremely foolish or ignorant to think that RISC-V won't undermine most ARM-licensed devices. The vast majority of places where ARM is used (even in Apple devices) is small and cheap microcontrollers that mostly cost whatever their license demands. If Apple could switch their internal ICs to RISC-V, they would already be saving a lot of money without transitioning to RISC-V fully.
Have they?
I certainly think you have a point about microcontrollers.
I find it very doubtful RISC-v would supplant ARM on the high end anytime soon. The licensing fees are pretty peanuts compared to the cost of designing your own cores and even the big players have been struggling to come up with anything that could compete against ARMs designs (besides Apple of course).
Also it’s not at all clearer what incentives would Apple have to switch to RISC-V? Sure they could do it.. what would they gain though?
If there were a compelling advantage at some point, they could go for it.
1977 : 6502
1984 (+ 7): 68000
1994 (+10): PowerPC
2005 (+11): x86
2020 (+15): ARM
Because of that I don’t see a next transition happen before around 2040.
If it happens my $0.02 is on riscv. That’s a big if, though. The main CPU is getting less and less relevant relative to custom accelerators, and they’re in control of their own instruction set, so they may be able to fix a few bottlenecks while (more or less) staying on ARM.
It was essentially the same with Intel and Arm, plus the fact that they could save money and further integrate vertically.
(As I recall, but maybe I misremember, Apple claimed a big performance advantage for their high-end G5 Macs but in the real world it was a bit dubious.)
They have an ARM architecture license. They design their own ARM implementations from the ground up.
If something can be done to RISC-V designs that would leapfrog ARM and ARM does not want to incorporate whatever breakthrough that is into their own cores for some reason that will mean that the people who rely on commodity ARM chips or on licensed ARM cores will be out of luck.
Apple on the other hand would be free to incorporate that into their ARM implementation.
RISC-v is an instruction set. If any company made a better RISC-v core than Apple can make themselves they could either buy it from them or make their own faster ARM core.
That’s what happened with Intel and PowerPC…
What improvements in the instruction set itself could make RISC-v fundamentally faster than equivalent ARM chips?
My sample includes Windows, Linux and Macs. My most recent intel mac was the worst laptop I’ve ever used, but it was the end of a run of machines that couldn’t reliably resume, had sub-2-hour battery life out of the box (advertised at 8+ hours), woke up to fry themselves in my bag, thermally throttled at random times, etc, etc.
“for M1 models, their ‘firmware’ update also brings a new Recovery system which is based on the latest macOS, in this case 12.0.1 even when the update is 11.6.1”
You can install Rosetta yourself from the command-line with softwareupdate --install-rosetta
Same thing happened with the PPC transition; 10.4.5 to 10.5.8 had a lot of PowerPC code even without Rosetta installed.
The "personalization" is basically firmware updates specific to your Mac. These are a lot bigger and more complicated on Apple silicon than on Intel. You can see this with the --downloadassets argument to the createinstallmedia tool:
https://scriptingosx.com/2018/10/include-assets-in-external-...
The same thing happens with iOS updates.
I know, I didn't say otherwise. I said the system is universal.
I wasn't even talking about the updated mechanism.
There are plenty more.
“According to the logged in iCloud account this user likes reggae, thriller movies, and paint programs. So… don’t patch flaw X7b-9?”
Still, a binary diff update would be helpful, especially since Apple seems to have more control over the system files. Maybe reproducible builds would help so you wouldn't end up with meaningless diffs
Thanks for the writeup.
It makes sense that maybe the x86 stuff might be there just in case of rosetta needing to do something. No idea but seems plausible that they want to have the fork there.
there definitely is some kind of per-instance registration of which laptop has which version, iphones similarly have a very visible "requesting update" state. Apple has secure boot on all their shit for a long time, they have hardware attestation and the OS will report when it's been updated, and they can monitor attempts for rollback attacks unless the laptop has been Startup Config > Security policy changed to allow insecure mode and older/non-signed/non-apple OSs or custom extensions etc. Apple legitimately did build a very secure default system, you can unlock it but you do have to go out of your way to go into bios etc, by default they will very aggressively assume that apple signing = on and version go forward.
Which if any of those files are bit-identical when received by different laptop identifier instances (via wireshark or w/e)? Like "it has to be downloaded from apple" is that because it's custom to your hardware instance?
Is there a common file image and then a mask, or is the second thing a giant blob of binaries that's the same for everyone, or what? Or just can't tell because it's encrypted or something?
Gigabytes of keys? I hope Apple isn't wasting tons of bandwidth to achieve information-theoretical security that's no more secure than than 256-bit AES in practice.