Hacker News Reader

I think currently M2 max is best bang for buck running interface in open source model. But use case is so niche that Apple probably doesn't actively start supporting open source models. In the long run I hope some smaller company gets shit together and starts competing with NVIDIA.

View on HN

The GPU support in ML frameworks however is really not impressive. I have a Macbook with M1 Max 64G RAM, I can load a 7b model for fine-tuning (Huggingface Trainer, Pytorch, MPS), but the speed is just too slow, can only reach to 50% the speed of an i5-12500 CPU in my tests.

At $6,000, how is M2 Max the best bang for the buck?!

One could get two used 3090s and setup a decent PC at lower prices.

Two 3090s don’t have 96GB of VRAM

That's true. But OTOH, one can't easily upgrade Macs.

With 96gb of VRAM, will you really need to? Personally I think apple servers might be adopted for AI and LLM workloads soon.

M2 max studio is 2000$.

Reply on news.ycombinator.com