Hacker News Reader

Comment by ttoinou | Hacker News Reader

M5 Max 64GB (sweet spot) or 128GB (only 1000 USD, better to keep it for the future) more are the best quality price ratio, future proof, reliable, resellable and flexible workloads. Harder to use as a server might be the only drawback

View on HN

What do you recommend for non-Mac setup? I am a Mac user, but its getting expensive, and not seeing reason to jump to the latest M5

Try looking into Ryzen AI Max 395. AMD made a CPU/GPU soc with unified memory specifically for ai inference. Can buy mini PCs with up to 128gb ram.

Isn't CUDA/nvidia the go to solution for most local models, with the rest being second class citizents?

Depends. ROCm is pretty well-supported for example.

Non-NVIDIA backends tend to get less support and new features land slower, or features that are expected to improve performance wind up hurting it instead. That sort of thing.

For basic “token in/token out” workloads without fine tuning, it’s probably fine ??

The Ryzen AI Max 395 128gb is super cool, but not fast for inference. Order of magnitude slower than dedicated GPU but at half the cost. You can run larger models on it but it's slow. Great for local async work. Not great for daily chat or code agent driver.

The latest NPUs are pretty fast, I think what is missing is more optimised software support.

The vRAM bandwidth is at least as much a problem as compute on these ones, there is a lot of data to shuffle around

Probably a comparable non-Mac setup will be Threadripper, but it will become much more expensive. My view is that actually Apple products are the cheapest on the market when it comes to performance.

M5 Max 128GB for $1k?

The memory upgrade is $1k on a Macbook Pro. The laptop is ~$5500.

I think they mean the upgrade to 128GB is +$1k.

Reply on news.ycombinator.com