Hacker News Reader

Comment by maxbaines | Hacker News Reader

Looks like a pretty useful offering, 128Gb Memory Unified, with the ability to be chained. IN the Uk release price looks to be £2999.99 Nice to see AI Inference becoming available to us all, rather than using a GPU ..3090etc.

https://www.scan.co.uk/products/asus-ascent-gx10-desktop-ai-...

View on HN

All Sparks only have a memory bandwidth of 270 GB/s though (about the same as the Ryzen AI Max+ 395), while the 3090 has 930 GB/s.

(Edit: GB of course, not MB, thanks buildbot)

I believe you mean GB/s?

The 3090 also has 24gb of ram vs 128gb for the spark

You'd have to be doing something where the unified memory is specifically necessary, and it's okay that it's slow. If all you want is to run large LLMs slowly, you can do that with split CPU/GPU inference using a normal desktop and a 3090, with the added benefit that a smaller model that fits in the 3090 is going to be blazing fast compared to the same model on the spark.

Eh, this is way overblown IMO. The product page claims this is for training, and as long as you crank your batch size high enough you will not run into memory bandwidth constraints.

I've finetuned diffusion models streaming from an SSD without noticeable speed penalty at high enough batchsize.

At that price (roughly 4000 USD), one could build a full HBM powered Xeon system from the Sapphire Rapids generation.

Either build a single socket system and give it some DDR5 to work alongside, or go dual socket and a bit less DDR5 memory.

I would hold my horses and see if the specs are actually true and not overblown like for the spark otherwise there are better options.

And if waiting six months is possible, do that.

Asus make some really useful things, but the v1 Tinker Board was really a bit problem-ridden, for example. This is similarly way out on the edge of their expertise; I'm not sure I'd buy an out-there Asus v1 product this expensive.

This is a Spark, so it is not going to be any different.

Reply on news.ycombinator.com