Hacker News Reader

> but I haven't seen any indication that training was quant-aware

readme on huggingface says they've benchmarked the quants -- for 17GB quant reported 1% avg loss across 15 benchmarks (sadly no breakdown).

I assume that's strong enough signal for QAT. Not just first party quants, but they cared to monitor degradation.

View on HN

> sadly no breakdown

That's exactly the point. We know short context knowledge stuff does not regress with quantization. But I expect agentic intelligence to suffer greatly.

If I were to pick one bench, I would like to compare quants on TerminalBench Hard. But then Glimmer already loses to 3.6 27B on it by a large margin.

Reply on news.ycombinator.com