Hacker News Reader

Comment by joshstrange | Hacker News Reader

A max-specced M3 would give you 128GB of ram vs 48 total from the 2 RTX cards and you'll still need a computer (CPU/MB/RAM/case/fans/etc). They might even out in price but you have more ram with the M3 and a way lower power usage if you care about (I know some people don't), not to mention it's portable.

If all you want to do is AI/ML then yes, a M3 is not your best option but if you want an extremely powerful laptop that can run AI models locally fairly easily (and it's well supported) then the M3 is a great choice. I love being able to download and test out AI models on my M3.

View on HN

Apple is well supported in that it's "better than everyone except Nvidia", and that's really only because a bunch of well known boutique developers seem to like the high quality of apple laptops and refuse to budge about mainly developing on them.

Still, trying to do significantly fun or interesting things in any major AI ecosystem, such as the Stable Diffusion one through Automatic1111 or with the LLMs in Oobabooga (which supports nearly all LLM backends i.e. llama.cpp), will be mostly crippled and stuff will break in ways that simply don't happen to Nvidia hardware. I'll take the 2 4090s all day, because I know that quantization techniques which shouldn't even be possible (who knows maybethe fabled 1 bit quantization seems inevitable at this point) will make it possible for me to stuff even the most bloated LLM into my measly 48 GBs will be available on Nvidia first and Apple (maybe) second.

I bring this up all the time and get downvoted by apple defenders (usually I’m one). Apples policy of locking out 3rd party gpus might be the cause of a serious decline in Mac OS over the next decade.

That or devs will start building Linux hosts for their gpus at home.

That doesn't really make sense, if anything the importance of having third party cards is diminishing. Apple Silicon is already massively competitive at running things like Blender and Video Compositors. And the trends are towards apple investing in improving gaming and ML workloads (which they've made large strides towards this year), and the relative attractiveness of other GPUs is going down. Almost all of the signs are pointing at apple making their hardware the place to run on device ml.

Mac Studio Pro's with up to 192GB ram are an alternative here too though still on the M2 Ultra chips if you don't need the portability. Not sure how relevant the generation bump is for running the models.

Might be a good hold-over for X years until the consumer hardware catches up and/or the model optimizations make the same hardware perform up to today's DC hardware.

I assume the Mac Studio Pro must be getting an M3 Ultra refresh in 2024? I wonder how well linux run on that.

Reply on news.ycombinator.com