Hacker News Reader

Comment by EnPissant | Hacker News Reader

Strix Halo is pretty useless for inference because the prefill is too slow.

At least this thing is actually useful, and there are $3k variants available.

I keep reading comments saying it's useless from people who clearly haven't actually used it.

I'm building and using this machine daily for building and using applications with LLMs, TTS, STT, ASR, and image generation.

Which, GB10 or Strix Halo?

Pretty sure they mean the GB10

Nope, I was referring to Strix Halo

Oh, well awesome. Glad to see you are getting so much out of the Strix line. I am eagerly awaiting the next gen. I think that will be a tipping point in AMD’s favor. I am a bit of an AMD nerd, even though they don’t seem to love their developers as much as Nvidia.

Before anyone gives me grief my company has a strategic partnership with Nvidia, I do AMD under the cover of darkness. So I live in both worlds. I’m a bleeding heart for the under dog…if being a 360B market cap company makes you the “under dog”.

Strix Halo

I don't believe you. It has very poor compute.

Are you basing that on your informed first hand experience, or based on your assumptions backed by no actual experience using the hardware?

I don't know what you want me to tell you, you're welcome to believe whatever you want but that doesn't change the reality I experience actually using the thing.

Benchmark numbers and first hand reviews are readily available if you bothered to look.

I am basing it on benchmark numbers. It's compute is just too poor to be useful for LLMs or Image generation.

For example: For LLMs, it's easy to do the math, and see how long you will be waiting for 50k input tokens.

Yeah, there is a lot of advantage to having this machine because the CUDA stack is still king. My Two AMD GPUs are suffering when it comes to working with ROCm stack. I have forks of Ollama and VLLM that took many weekends to figure out.

If you're on Strix Halo, check out Donato's prebuilt toolboxes for ROCm with RADV or Vulkan:

https://github.com/kyuz0/amd-strix-halo-toolboxes

It takes all the work out of it, you just start llama-server in the container context and you're off doing inference without having to figure out dependencies.

Oh yeah he is doing great things. Not on Strix myself but his dual AMD AI Pro r9700 build ironically is the same machine I built.

Ah I see ASUS machine for 3K. Hopefully in a year or two we can get a better machine with twice the ram for same price. Then I’d probably buy one.

Reply on news.ycombinator.com