External boot disks still don’t work properly with M1 Macs
eclecticlight.co
eclecticlight.co
When you boot into the "Startup Options" menu, you are booting into a special macOS partition in the internal SSD. M1 Macs use macOS as the moral equivalent of the UEFI setup menu. That boot picker that looks like the UEFI boot picker on Intel Macs? Yeah, that's a full-screen app on macOS made to look similar.
So how does external boot work?
The "blessing" process done by the Startup Options screen involves copying the entire macOS Preboot partition - iBoot2 OS loader, Darwin kernel, auxiliary CPU/device firmwares, device tree, and some additional stuff - to the internal SSD. It then creates a local, signed boot policy that allows system firmware to boot this macOS install.
You aren't booting from an external disk. You are booting from the internal SSD, with the root filesystem on the external disk.
Additionally, the integration of the macOS user credentials with the SEP means that you can't "just" take an install made on another machine and use it on a separate one. It involves importing user credentials into the local machine. This process isn't implemented properly yet.
As you might expect, this entire mechanism introduces a ton of corner cases around updates, boot selection, etc., and it is still very buggy and broken. The M1 launch was, when you look at details such as this, very obviously rushed.
Further reading for those interested:
https://github.com/AsahiLinux/docs/wiki/M1-vs.-PC-Boot https://github.com/AsahiLinux/docs/wiki/SW:Boot https://github.com/AsahiLinux/docs/wiki/SW%3AStorage
I'm actually eagerly waiting for Apple to fix this in future system firmware releases, because my plan for installing Asahi Linux for end users with a minimal amount of fuss is to abuse the mechanism to adopt foreign macOS installations. This elides the current need to have a completely wasted, 60+GB macOS install as a dummy to actually launch Linux (we could clean it up and resize it to get the space back, but it still makes for a very annoying install process that I'm hoping to avoid).
On the plus side, this dual-booting mechanism is very cleverly designed to separately secure different OSes, so you do not need to downgrade security on your machine to install an unsigned OS like Linux. It is a separate OS with a separate security policy. You can keep your macOS install fully secure, and capable of running iOS apps and other actions that require secureboot, and install Linux (or another macOS with a custom kernel, kexts, etc) in parallel, completely unintrusively. So kudos to Apple for designing this whole thing, and for opening it up for us to use :)
So this means that the life of all M1 Macs is locked to the life of the internal storage, as the (inevitable) drive failure will render the whole computer unusable?
Strictly speaking, the things can boot off of DFU (USB device mode) too, but to make that useful for regular boot you need to ask Apple, as currently you cannot boot a normal OS like that as far as I know, only their signed restore bundles (which is how you fix an M1 Mac if you wipe the SSD).
It is being nitpicky on my part, but I think we're reaching the point where these machines could be expected to last long enough for SSD weardown to become a real issue, even with current flash tech. While this has theoretically been a problem for the past 5 years now, it's a bit disappointing to learn that now you can't use an external drive in the event of the (irreplaceable) internal drive failing.
Various crap bolted on by AMI and the OEMs… well, YMMV – remember these laptops where someone did rm -rf with mounted efivarfs on Linux and that bricked it because it just refused to boot without the variables :D
Not that it (worn out flash) is gonna be a problem in practice, anyway. Neither will the M1's inability to boot without the internal drive. Like Marcan said, it's not really likely to die in realistic use within a pretty long lifespan, I just find these sorts of shortcomings/regressions ... disappointing. I get the various reasons why, it's just annoying.
[0] The low tier 2016 and 2017 touchbar-less MBPs do, quixotically, have removable storage, but using a proprietary format vs the NVMe sticks in previous models. All touchbar MBPs (and M1 Macs) have soldered flash chips, and the TB-less ones were totally discontinued in 2018 IIRC.
Anyway, thanks for doing what you do there <3
Furthermore I don't see any reason to think that OP's issues are to do with the hardware or that we will have to wait for the next hardware model to have these issues resolved. It seems like a simple OS-level software bug to me.
EDIT: According to other posts there are in fact some volumes on M1 Macs where if deleted, you will need to do a DFU mode recovery. So perhaps don't actually delete all existing volumes on an M1
However I still don't think DFU mode was necessary in your case. If it were necessary, then you would not be able to boot the system at all with the normal method.
Trashing just your OS partition should not require DFU mode. Although yes, it seems like trashing the new boot data partitions might.
And those aren't out yet, those would be the 16 MBP, the Mac Pro, the iMac and iMac Pro, and so on.
If that isn't a cpu-demanding task then I don't know what is.
(not trying to be rude, just that the Air is currently a very capable and pro-tier machine)
[1] https://www.macrumors.com/2020/11/17/apple-silicon-m1-compil...
If you're a programmer that mostly relies on multicore performance, the current M1 Macs aren't the gold standard. Unless of course you are drawn to the OS or something else.
Not really, since most of those kinds of professionals concerned already used Intel Macbooks. It's not like they're gonna win the "build some tower yourself crowd".
Depending on where you live, there may be some supply issues, especially if you are looking for a model with an NVidia 3000 series GPU.
Unless you're really being pedantic the two are of the same form factor. Maybe you are comparing the Intel ThinkBook 14? That one is significantly heavier and a bit thicker.
If an increase in size of around 5% is too much, there are smaller laptops with the same processor family.
We're talking here about pros in the context of "increased hardware demands".
This has little to do with "Pro" in the product name or not, nor with them being more professional than other working people.
I mean, really, it just means "the expensive one", as in AirPods Pro, but sticking to Macs for a second: if your job is video editing, or publishing, the bigger and nicer screen is going to trump a faster processor and better battery life. The (much) better speakers are also a big plus.
Smart users in that category are holding off new purchases, if they possibly can, until the M2 (presumably) 16" MBPs drop. Especially if the rumors hold and it comes with an onboard SD card reader.
Otherwise it's entirely possible it's for security reasons and somehow related to the secure enclave stuff.
But I might way out of my depth here.
What is the use case for using an external boot disk with a mac?
And I’ve booted Linux from an external drive a zillion times on a PC. It’s fairly handy to have the option.
Simple things like USB not having any decent way to do shared memory with the host are a big part of it...
I used to support a similar need with an enterprise tool that cost exponentially more to implement and support.
* Booing up a disk from another machine where it's suffered a non-disk failure.
* Because I want to.
I'm not saying that it's "normal" not to be able to boot from external devices, I'm just answering your point
Safer to use a separate disk or partition.
It's probably simply unimplemented, and rather than try anyway there is a check somewhere in the firmware that the drive being marked bootable meets some criteria. Clearly the internal drive meets that, and some external drives do. Running a tracer on the commands sent to the drive would likely identify it.
- Continuing to be able to use your laptop when the internal drive has failed. Very helpful if you can't afford a replacement, or if you need to carry on without waiting for a replacement and you have a recent backup.
- Booting older versions of MacOS (that don't know about APFS, or if you have reason not to trust APFS partitioning) for testing, compiling for an older target, running software that doesn't run on the current version of MacOS, or other reasons.
(But old MacOS won't work with the M1 so that doesn't matter at the moment.)
But is anyone else shocked at how well the transition to ARM has gone for Apple? Because I am. I figured this would be something I would avoid as long as possible but honestly I'd probably be happy with this year's MBP M1 refresh.
I totally get why some folks would want to wait for the next-gen of these initial machines, but I'm not surprised at how well it's gone.
I chose to buy the initial M1 MacBook Air model right-off-the-bat. The reason for this is that I went through the last two Mac CPU arch transitions (68k -> PPC, PPC -> x86), and the way Apple was able to handle those was truly impressive to me. Plus I really needed more built-in storage ASAP :)
Also the kernel (that I thought was the actual kernel in use) in /System/Library/Kernels reports "Mach-O 64-bit executable x86_64" so I'm kinda confused about all this.
I'd appreciate any pointers.
The installation should be bootable by both, but not sure that the flow to bless an install done on another machine to be bootable on an other M1 one is done/fully functional yet. (on M1 machines, you need to bless external volumes before booting from them)
Booting an install done on an M1 on an x86 Mac should work already though.
The built-in bootloader actually boots iBoot2 from /System/Volumes/Preboot/(UUID)/boot/(long hash)/usr/standalone/firmware/iBoot.img4, and that then loads the Darwin kernel from /System/Volumes/Preboot/(UUID)/boot/(long hash)/System/Library/Caches/com.apple.kernelcaches/kernelcache.
However, the system firmware can only boot this from the internal SSD, not from any external storage. When you choose an external disk to boot from (the "bless" thing), those files get copied to the internal SSD, and it boots from that instead.
You can then rebuild your own kernel cache from /System/Library/Extensions and /System/Library/Kernels. (eventually with a recompiled kernel)
Go to https://apps.apple.com/app/macos-big-sur/id1526878132 (for Big Sur), click "Get", use Software Update to monitor the download progress, then "rescue" the installer ("Install macOS Big Sur.app") from Applications folder.
They aren't going to backtrack on all that. That'd be silly.
So please stop the FUD about Apple locking their devices down; this is a Mac, not an iPhone :-)
Has nothing to do with trying to impose additional restrictions.
For now, maybe. Consider the time when microsoft was partially successful to boot-lock arm devices: https://www.softwarefreedom.org/blog/2012/jan/12/microsoft-c...
The old adage "do not assign to malice what can be properly explained by stupidity" should not be blindly applied when it involves powerful companies.
Bugs are being worked out slowly, and there's no reason to assume bad intentions here. (I have an Apple M1 MacBook Air, and it boots from a Samsung T7 external SSD just fine)
Though ARM appears to be missing an open-spec BIOS/Boot standard like x86 has.
It's just the embedded junk (which includes Apple) that doesn't care. Apple has vertical integration, they only care about the full experience including their own OS. Embedded SoC vendors only care about having some "BSP" fork of Linux because that is enough to make a crappy device with it. There's just, unfortunately, zero motivation for these vendors to embrace standards.
One thing I do recommend though: remove airpods from the Apple account before cloning and pair again once the same system is running on new hardware. Sometimes this causes problems
It's very useful to be able to seperate data from the hardware. If something is wrong with a machine I can just plug straight in to another one. An SSD is much easier to carry than a laptop.
The other problem with internal hard drives is when they get stuck inside e.g. with MacBooks being sealed these days. External SSDs can offer great flexibility and a trade off worth considering. I can imagine people having security concerns etc.
Disappointing to hear this about the new M1s.
I recommend people interested in dipping their toes into Linux Gaming trying this approach:
This was one of the touted technical uses for the first (Firewire) iPod. Apple had a notion that people could carry an iPod with them, and then boot any Mac from it and start working right where they left off.[1] This is why Xserves had a Firewire port on the front.
[1] As a point of interest, this used to also exist in the radio industry. Certain brands of control boards had "personality cards" that could be tuned[2] to a particular D.J. Then when the D.J. had to work from another studio, or remotely, he could pop his personality card into the control board and it would be preset to him.
[2] And by "tuned," that meant tuned for his voice[3], to make it sound the way he, or his program director, wanted it to sound.
[3] Even more interesting: Though these were circuit boards, the tuning was done mechanically, and by hand. They were a series of potentiometers covering different frequency ranges.
As those are equivalent with 4 PCIe 3.0 lanes, exactly like the internal M.2 type M SSD slots, there is no performance difference.
Only the latest laptops with Tiger Lake, and the laptops with Ryzen 5xxx to be introduced soon, have PCIe 4.0 M.2 slots, so they will be able to have faster internal SSD's.
I expect that the interconnection standards for external devices will also continue to be improved, restoring parity with the internal devices.
Can you suggest some brands or models?
I have an NVME 2.0TB drive in a USB-C enclosure and while fast as an external this is not fast enough for booting and running the dozen or so apps I keep open on my MBP.
YMMV if you don’t act as a power user.
Keep in mind that USB-C != Thunderbolt 3. Yes, the latter also physically connects to a USB-C form port, but it's totally different. It's quite confusing and sad.
To address any conventional SSD vs NVMe SSDs performance gap maybe consider and external NVMe SSD!
PCIe costs you less than a microsecond of latency. A good SSD has 60 microseconds of latency. You're not going to notice any difference from moving the controller.
Many people still use HDDs for game storage.
This sounds like an ad, but Sandisk has USB 3.2 flash drive with NVMe inside, that according to this has up to 2 GB/s read speed: https://www.amazon.com/SanDisk-Extreme-Portable-External-SDS...
I used the version 1, not NVMe, which can only do half that speed, but I found it plenty fast.
Not that that means anything for performance. NVMe just means that it uses PCIe.
Since the external port is USB, the internal use of NVMe is actually a downside. The drive actually has two separate circuit boards inside. One of them takes up space and power just to convert NVMe to USB, and wouldn't exist in a better design.
https://i.imgur.com/6oPiOls.jpeg
https://static.tweaktown.com/content/9/2/9280_08_sandisk-ext...
(Alternatively, if you really can't switch job, just buy an ssd yourself)
But how do you prevent the external SSD accidentally becoming unplugged when you're doing this with a laptop? And what would actually happen if that did occur?
Currently this is the weak point of using external bootable SSD's. Because of that, I prefer the SSD's where the Thunderbolt/USB cable is captive at the SSD end, as then there are less connections that can be unplugged accidentally.
What I would like is to have in the laptop a CFexpress memory card slot.
The CFexpress memory cards have a PCIe electrical interface, but they are mechanically identical with the older Sony XQD cards, i.e. they are solid, rugged, not flimsy like the SDXC memory cards, and they allow a very large number of insertion/extraction cycles.
If the laptops would have such card slots and if SSD's would be available at a reasonable price in this format, then there would not remain any reason to use non-removable SSD's. The current CFexpress specification is based on PCIe 3.0, but future versions can be updated to PCIe 4.0, like the internal M.2 SSD slots of the latest laptops.
Thinkpads up to 2013 also had a tool-less removable disk drive that allows for hot swapping 2.5" SATA drives in seconds while keeping them safe inside. This has been abandoned with the obsolescence of optical drives though.
> The CFexpress memory cards have a PCIe electrical interface, but they are mechanically identical with the older Sony XQD cards, i.e. they are solid, rugged, not flimsy like the SDXC memory cards
I agree, though with SDExpress becoming standard soon there is a much realler possibility of future laptops having pcie microsd slots, which might be good enough if you can find a durable model.
[1] https://thinkmods.store/products/expresscard-to-nvme-adapter
For those not using Apple laptops, this might be a thing, but I still doubt it's something manufactures will see a lot of demand.
My favorite has been the USB memory sticks that are the same physical size as a bluetooth dongle. Since they are so low profile, the chances of them coming unplugged are pretty damn slim. The only draw back was they are pretty small in capacity for trying to run an OS.
A gentle reminder that you live in a world where SSDs on some laptops are not removable at all, as are batteries. And I’m leaving out minutiae like RAM.
I’m just not seeing it as possible, how would you plan the obsolescence of such devices?
Yes, it's a shame you can't charge a 10x markup on the storage the end user takes with them, but you can still charge a 10x markup on the built in storage that's non-removable and whose failure still blocks booting (necessitating replacement of the whole thing, naturally)
https://www.crowdsupply.com/eoma68/micro-desktop/updates/sta...
What I’ve also seen some companies do is simply duct tape the ssd to the laptop lid so they can walk around freely to meeting rooms etc.
At home I don’t need that
Perhaps useful for dual-booting between personal and work installations, considering MacBooks are sealed with one drive?
This way I kept all the company settings and programs working in a way that a time machine restore couldn’t do.
Works just the way you would expect from an internal ssd and switching takes less than two minutes if necessary
I have been doing this for decades, but I do it with USB sticks, not SSDs, with NetBSD, not Windows. After boot I can pull out the stick, freeing up the USB port for something else.
I would be interested if I coul do the same with Windows but it does not seem as straightforward. I know it is possible but I also know there is a good chance I would never get it to work.
For example, the EasyUEFI link you provided suggests I need to purchase software, the software only runs on Windows, Windows 7 is not truly portable and the authors are not clued in to the idea of separating data from hardware:
"We recommend that you use Hasleo BitLocker Anywhere to encrypt Windows To Go drive to keep your data safe. If you want to upgrade Windows To Go to Windows 10 October 2020 Update (Windows 10 20H2), please go to Windows To Go Upgrader."
If only there was a way I could buy Windows on a stick instead of having to buy a computer with pre-installed.
Does this mean the entire OS is loaded into memory? Is this a NetBSD thing?
But on anything ARM based? The whole ecosystem doesn't give a fuck. Everything is hyperspecific for one platform exactly. At one point, the Linux kernel had thousands of board source files that specified exactly what hardware your cursed ARM board had. Then they invented device tree but frankly the situation has barely changed, don't expect to ever build a generic ARM image for anything.
Not every machine is a guaranteed boot. It depends on MBR/GPT etc and whether UEFI is disabled in the BIOS and other configurations. However generally yes, they will. I've even have setups where the SSD would would have macOS/Windows/Linux. Much older motherboard pre-2012 generally don't like this setup and it can be v.slow.
For example, with bitlocker, won’t you need to enter the recovery key when trying to boot from a new machine? And have to sign out and back in to all relevant OS level accounts? Even then I face authentication issues at times
I really would like this to work seamlessly because moving my internal SSD work disk to an external one would be far safer than lugging it around inside my personal laptop all the time. But the work disk has to be encrypted...
Also, for hardware compatibility’s sake, I’d think Linux would be a far superior daily driver OS to ‘multi-hardware boot’, considering relevant drivers are loaded from kernel on boot rather than selectively pre-installed at OS creation time for that one device, increasing plug and play compatibility
At work due to remote work I cannot have fully encrypted disc with Windows as I have to reboot remotely. So I left a small enough partition for Windows, then created another partition for my data that I encrypted with strong password in Bitlocker. Then I symlinked my user directory from C:\Users\UserName to a directory on D: and created an extra account that I use after reboot to unlock the encrypted disc with my data.
This is not ideal, as Windows still may store my data on C:\, but if one disables virtual memory, it is a reasonably secure setup.
But try move a bootable bitlocker encrypted disk to new hardware and you’ll have to enter the recovery key
I would really like to be wrong about this since it would make my life much easier, but this understanding is based on experience using multiple work machines with encrypted boot drives every day :(
this is not true. you can configure bitlocker with or without TPMs.
just google it. also the doc for the powershell command talks about it in the establishing a key protector section
https://docs.microsoft.com/en-us/powershell/module/bitlocker...
I used the `manage-bde` command rather than powershell:
https://docs.microsoft.com/en-us/windows/security/informatio...
The GUI for bitlocker doesn't provide access to all the functionality that manage-bde provides (iirc: if a TPM is present, the passphrase options aren't presented in the GUI. And it used to talk about a "PIN" instead of a passphrase/password, but the "PIN" can (with some gpo tweaking) contain letters/space/punct as well as numbers.
And you mention gaming setup, assuming windows, is this as portable as you make it sound? No weird hardware-dependent issues?
I could either boot the Windows install directly from the SSD, or I could boot Linux and then run the same install in qemu, without issue.
I think there used to be more problems if you changed the hardware too much from the original install. Like, if you changed your motherboard (which the above would do of course), then you couldn't run the same install. But I don't think that's as much of a problem anymore?
This was of course not an external SSD, so I don't know how that changes things, if at all.
Nice to hear that it is possible. I feel like MS essentially gave everyone the wrong idea about OSes dealing with hardware changes.
If the disk is plugged into an already booted system and the OS doesn't recognize the filesystem format, I believe macOS and Windows display a prompt to format the disk, which someone may accidentally click.
I would have assume that this would not work unless the hardware was the same - due to device drivers, etc.
I should probably mention I just stumbled on to this approach rather than having forged it through clever thinking -- it was originally due to a faulty motherboard burning out internal HDDs consitently. So I HAD to opt for an external drive and once I did the going back made little sense as it just imposed restrictions I didn't want anymore.
For such scenarios common by media creators external disks fail to “just” plug. Some apps uses dongles or allow activating to usb sticks. But still it’s the exception.
But I mirror my internal NVMe SSD to external SSD as the performance benefits of the former matters for me most of the time and when needed bootable external SSD is available at disposal.
On macOS(not M1), Carbon Copy Cloner could create bootable clones of the disks. But I'm out of macOS for good as changes in Big Sur was too much for me, No offense to those who like it; but it seemed like a unicorn poop for me. Mojave was the last version which preserved the essence of macOS IMHO.
Now I'm back to Linux, hopefully btrfs(snapshots) + Timeshift should recreate the same workflow.
Load a Windows 10 ISO and click Windows to Go, wait a couple minutes. Full Windows on a USB and it’ll automatically rediscover hardware if you move it to a new PC.
I have worked with External SSDs on Macs for years. Partially because to circumvent the Apple Price Tax.
But also to just treat storage as something easy to swap.