Hacker News Reader

Parent Full thread

vardump

3060 doesn't have 128 GB RAM.

View on HN

128GB / 12 GB = ~11, * 200€ = only 2200€ plus mining rig mobo.

It would be cheaper to buy up a dozen 3060s and build a custom PC around them than to buy the Spark.

Except the Spark was designed to have everything nicely working.

And as this post shows, it doesn't.

More than most AMD stuff.

It will be really hard to put a dozen in a single PC. Then to connect them at good speed. Add to that a few new power lines to feed it all.

True, and overkill too. With DDR5 partial offloading it would only take probably two or three at most to outperform it in every metric except power draw. My point was more that the pricing is absurd for the performance.

I agree on pricing. But it includes 'free' software. Similar model Apple has. You don't buy just hardware.

And a 14B model running at 22tg/s means you won't be using that 128G RAM for inference either.

Yeah I’m honestly unclear on Nvidia’s thinking here - inference speed is unbelievably slow for the price.

Given the extreme advantage they have with CUDA and the whole AI/ML ecosystem, barely matching Apple’s M-ultra speeds is a choice…

Definitely a choice to give it low memory bandwidth. Probably to avoid customers thinking it can replace any data center GPU for inference use-cases.

You can use all 128 GB if you use a MoE model

Reply on news.ycombinator.com