Hopefully this makes it really nice for people that want the experiment with LLMs and have a local model but means well funded companies won’t have any reason to grab them all vs GPUs.
What this does offer is a good alternative to GPUs for smaller scale use and research. At small scale it’s probably competitive.
Apple wants to dominate the pro and serious amateur niches. Feels like they’re realizing that local LLMs and AI research is part of that, is the kind of thing end users would want big machines to do.
Also, I’m curious and in case anyone that knows reads this comment:
Apple say they can’t get the performance they want out of discreet GPUs.
Fair enough. But yet nVidia becomes the most valuable company in the world selling GPUs.
So…
Now I get that Apples use case is essentially sealed consumer devices built with power consumption and performance tradeoffs in mind.
But could Apple use its Apple Silicon tech to build a Mac Pro with its own expandable GPU options?
Or even other brand GPUs knowing they would be used for AI research etc…. If Apple ever make friends with nVidia again of course :-/
What we know of Tim Cooks Apple is that it doesn’t like to leave money on the table, and clearly they are right now!
Theoretically they could farm out the GPU to another company but it seems like they’re set on owning all of the hardware designs.
SJ loved to quote Alan Kay:
"People who are really serious about software should make their own hardware."
Qualcomm are the latest on the chopping block, history repeating itself.
If I were a betting man I'd say Apple's never going back.
I guess there are other kinds of scientific simulation, very large dev work, and etc., but those things are quite a bit more niche.
I think Apple is done with expansion slots, etc.
You'll likely see M5 Mac Studios fairly soon.
I think the hold up here is whether TSMC can actually deliver the M5 Pro/Ultra and whether the MLX team can give them a usable platform.
That said, the need for them also faded. The new chips have performance every bit as good as the eGPU-enhanced Intel chips.
My suggestion is to accept that format and just provide a way to network them at a low level via pci or better.
This is an issue for some industry-standard software like CUDA, which does provide BSD drivers with ARM support that just never get adopted by Apple: https://www.nvidia.com/en-us/drivers/unix/
Having used both professionally, once you understand how to drive Apple's MDM, Mac OS is as easy to sysadmin as Linux. I'll grant you it's a steep learning curve, but so is Linux/BSD if you're coming at it fresh.
In certain ways it's easier - if you buy a device through Apple Business you can have it so that you (or someone working in a remote location) can take it out of the shrink wrap, connect it to the internet, and get a configured and managed device automatically. No PXE boot, no disk imaging, no having it shipped to you to configure and ship out again. If you've done it properly the user can't interrupt/corrupt the process.
The only thing they're really missing is an iLo, I can imagine how AWS solved that, but I'd love to know.
Macs are a minority in the datacenter even compared to Windows server. The concept of a datacenter Mac would disappear completely if Apple let free OSes sign macOS/iOS apps.