Hacker News Reader

What type of hardware, GPU are you utilizing? I have the same model self hosted at home

1x MI300x , rocm/sglang , highly optimized image, FP8, FP16 Cache (this fixes looping issues with qwen3.6-35b

~100tok/s average