Hacker News Reader

This is a serving bug or quantization issue. I had all kinds of issues that were like this on DGX Spark until I found a single-GB10 vLLM recipe [1] that uses Nvidia's NVFP4 quant. The community quants did not work well.

Another failure mode you may see is inordinately long CoT. Properly served, the model is good at calibrating its CoT length to the difficulty of the immediate task.

[1] https://github.com/blazux/qwen3.8-Flash-DGX

View on HN

I get the same messages in a MLX Q8 quant - I assumed it was reacting to the harness, but it doesn't seem to cause issues.

I've experienced the same, IIRC also using NVIDIA's NVFP4 quant. Also just decided to ignore because it didn't seem to cause any real issues. I figure it might be a training thing, since the hallucinated user messages seem to occur immediately after tool calls or when it's checking its work.

I’m using Eugr’s recipe (TP=1) with Nvidia’s NVFP4 quant and I’m still seeing it sometimes! Rarely. But sometimes

Still works great though!

Reply on news.ycombinator.com