Author here. The short version: Qwen3.8's 248k-token vocabulary contains scraped-junk tokens that got almost no training (embedding norms down to 0.001 against a vocabulary mean of 0.899). Put one in a slot that matters, like a username, order ref, or SKU, and the model swaps in a hallucination while keeping the output schema-valid. An order ref of "Kinhted" came back as "order_id": "shelled", 16 out of 16 times at temperature 0.7.
We ran six control models. Mistral and Llama have the same class of problem and lose 5 to 7 of their 12 lowest-norm tokens. Both Gemma generations are clean, so a curated vocabulary avoids this entirely.
Happy to answer methodology questions. If you want to know whether a specific checkpoint has this problem, name it and I'll run the scan.