Hacker News Reader

I'm getting around 27 tokens per second decode using vLLM + Qwen 3.8 27b nvfp4 + DFlash 2 on the DGX Spark.

I am working this out with Fable right now, for getting this running on my DGX spark homelab; it mentioned that there might be issues with the 'optimized LM-head restrictions' that unsloth NVFP4 ships with. Have you had any issues here?

Are you trying this with vLLM? Or a different engine? I am getting about 15 tok/s on my spark on my current setup using the 0.26 nvidia vLLM image and MTP.

If you are using the Unsloth nvfp4 checkpoint, you need to patch vLLM+DFlash 2 to accept the quant's FP8 `lm_head`.

Reply on news.ycombinator.com