Apple has good uniform hardware to enable this, but they are a product company and an “AI studio” would not fit their usual definition of a product.
I do hope they are considering going in that direction though.
I do hope they are considering going in that direction though.
1. Why would they miss the opportunity to go all in and make use of the Neural engines for this? Or do they already, and I just don’t know how to interpret it?
2. At what point is Apple going to think - hmm, we have a kick ass processor on our hands. What if we run our server fleet - the ones that serve iCloud, Apple Store, all Apple services, databases etc on M* processors? It sure would help even better the economies of scale for Apple to go to TSMC and say - here is our new CPU design for servers, iPhone, iPad, watch and whatever VR thing. Why no love for the server side that must be orders of magnitude power hungrier today?
With upgraded ram (for just 230€ for each 8GB) and storage (just over 1000€ for 2tb; a samsung 990 pro is like 170€) that might be another story, but "check your specs before downloading this app" seems very un-appley. Also, no cuda support etc. Maybe if they make it exclusive to the mac studio?
As for the price - when you get to 32/64/96GB levels of RAM it’s the cheapest setup on the market that can get you this much VRAM. At least that’s what it was half a year ago when I checked the last time
That is the point. Cuda is widely used in particular for training as opposed to tuning and is just flat out not available. So you buy your nice $6000 machine and it just does not work for its intended purpose.
> it’s the cheapest setup on the market that can get you this much VRAM.
shared VRAM is not everything.
Nvidia intentionally nerfs the amount of VRAM in their consumer cards so you need to buy their ridiculously overpriced enterprise cards. I think it's fair to say Apple's lineup is the cheapest way to get >64GB of VRAM-ish memory and that's definitely not because Apple prices their products so fairly.
Those are not equivalent in speed. macs RAM are much slower than these GPUs.
You can swap memory back and forth between RAM and VRAM (with a huge performance penalty) of course, but that's not exactly usable or comparable to what Apple's VRAM sharing setup allows.
Nvidia doesn't sell a nice-but-not-amazing GPU equivalent to Apple's processing power and memory bandwidth that's also capable of operating on >80GB of VRAM at once. Apple's SoC is kind of an oddball in that regard.
I suppose you could take a regular old iGPU (for AMD, Intel, probably also Qualcom/Mediatek) and use its shared memory capabilities as a comparison. However, iGPUs are terrible at machine learning tasks, they don't come close to what Apple can do with their dedicated accelerators.
The best middle ground may be the laptop GPUs with both dedicated RAM and shared RAM, but those will start swapping memory back and forth like crazy running large ML workloads so they're not really comparable.
If you want to run a model that operates on a huge amount of memory at once, I don't think there is a desktop option that can do what Apple does without going for the massive overkill GPUs that will crush the Macbook in terms of performance (at great cost).
Perhaps you know a GPU or iGPU that's capable of running 80GB VRAM workloads at comparable speeds? Because I don't.
- you need more than 16 or 32 GB of ram.
- but less than 100.
- speed is important.
- but losing a factor 4 compared to a GPU is fine.
- money is very limited.
- but paying 6000 for a mac studio is cheap.
- this is very important professional work.
- but I don't need servers, ECC ram, raid, another OS...
Don't get me wrong, I also think Nvidia is heavily price gouging, but there are definitely trade offs and it is not clear at all why 80GB is your magical number and not, say, 108 or 28.
Year 0 of cool thing - Nothing and silence
Year 1-3 of cool thing - Maybe, if you're lucky, a mention on some hardware thing related to it
Year 3-5 of cool thing - Hardware or software launches that uses thing, no mentions of this besides the earlier one if any.
Year 5-6 of cool thing - Next part of hardware or software launches that uses previous launch, no mentions of this besides the earlier one if any.
Obviously, the time-frames differ, but that's generally how they do things.