Hacker News Reader

It's not much worse RAM, though. RTX 4090 has memory bandwidth of 1050 Gb/s. M2 Ultra is 800 Gb/s. And you can get a Mac Studio with Ultra and 128Gb of RAM for $3K or less. It's great for 70-150B models.

You're correct that it's only good for inference, but most people running local LLMs only do inference.

View on HN

That config Mac Studio costs 5,800 Euros (minimum). Where do you get it for 3,000 USD?

You don't need an M2 for this, an M1 will do just fine. Nor do you need the one with maxed-out SSD, which jacks up the price considerably. Finding a brand new M1 Ultra for $4K or less right now is pretty easy. When I got mine, about a year ago, $3K was the best deal I could find.

Reply on news.ycombinator.com