AMD's game-changing Strix Halo, formerly Ryzen AI Max, poses for new die shots | Hacker News Reader

AMD's game-changing Strix Halo, formerly Ryzen AI Max, poses for new die shots

tomshardware.com

33 pointsrbanffy19 comments

>AMD also claims its Strix Halo APUs can deliver 2.2x more tokens per second than the RTX 4090 when running the Llama 70B LLM (Large Language Model) at 1/6th the TDP (75W).

That if true is wild.

It's because of the bigger VRAM - 70B parameters don't fit into the 4090's 24GB.

Wait, this claims 50 TOPS. How is it faster than 4090 that does >300 TOPS?

More RAM, so less movement of the weights around to generate a token. Most of the speed limit on a LLM is bandwidth of getting the weights around. To a great extent, your token speed is approximately your (model size)/(effective bandwidth). If you need to shuffle the weights into VRAM from main RAM, you halve your speed (bandwidth used both to move into VRAM and out). If you need to pull the weights from disk, even worse.

While true, the benchmarks are not run on the Ryzen's NPU but the much stronger GPU.

It's not quite "just give me a mini PC with the PS5 SOC in it", but it'll do.

You can (kind of) buy that: the AMD BC-250. It's the PS5 APU as a mining card. It can run Linux. Some people have even figured out to enable the extra cores, although they might be disabled due to faulty silicon.

https://www.phoronix.com/news/AMD-RADV-PS5-BC-250

https://www.techpowerup.com/forums/threads/someone-run-games...

You could slap one of those into an AMD 4700s system (PS5 APU with the GPU disabled) and build the most scuffed PS5 clone ever.

https://www.tomshardware.com/reviews/amd-4700s-desktop-kit-r...

Can the BC-250 cards be run as a cluster? They sound interesting and cheap enough (on eBay, at least).

I’m interested in such hardware in a steam deck

I don't think you're going to fit a 45W-115W chip in a Steam deck.

Could be wrong but I fully expect these chips to TDP-doen very very very well, as AMD chips have been doing since Ryzen/Epyc first came out. The performance drop off is far less than linear; efficiency climbs and climbs.

Even at 20W, I expect this would be an absolutely dominant gaming handheld, if someone was inclined to pack it all in.

But what really excited me is the idea that the Deck could Dock to a BFC, Big Fucking Cooler, that let it throttle up to 100+ watts. Good portable power, but then at home it could be a beast.

Sure, it'll likely throttle down, but at 15W it'll likely have similar performance to a 15W AMD mobile chip.

So you're running a chip that costs most of $1000 at the performance level of a $200 chip.

The thesis I'll try to state again more clearly is that bigger parts at lower frequencies can trump smaller parts at higher frequencies.

Efficiencies increase significantly as you downclock. Sure, a mobile part runs 15w. But it's relatively inefficient at doing so. If you want to drop it to 7w, now you're pretty efficient (Steam Deck lets you TDP down 15 -> 3w!!). But you are too slow for most games.

Starting with a big APU with gobs of ram bandwidth and downclocking that system is going to get you much better results from that 15w than using the small part at 15w.

I think the only thing its lacking compared to the base PS5 is memory bandwidth (256GB/s vs 448GB/s) but in exchange you get much lower latency. Everything else is a massive upgrade over the PS5. The cores are way newer, clocked higher, and you can get twice as many. The CUs are newer, clocked higher, and more numerous as well. It can be specced up to 8x the ram of the PS5.

Is there a workload where that 75% more bandwidth is more important than raw perf or access to more than 16GB of ram?

Yes, running LLMs.

I know that higher bandwidth is good for LLMs but are there actual workstation use cases where the speed outweighs the context and parameter limits imposed by 16GB of total system ram? I don't follow it that closely but it just feels to me like quality >>> quantity for laptops or mini PCs where an APU would be relevant.

Good to see they've dropped the "AMD Ryzen AI Max+ 395" naming. Their release announcements have been all over the place. This APU was the best announcement tho. Gives Apple Silicon a run for its money.

They have not. The title says "formerly" but the article correctly says "formally". "Strix Halo" is just the codename, "Ryzen AI Max+ 395" is a model name from that family.

Reply on news.ycombinator.com