This smells very badly of quantization - the extra commentary is a failure mode I observe frequently when dropping down from FP16 down to 4 bits.