Interesting.
If this were the case however, why would labs go through the trouble of distilling their smaller models rather than releasing quantized versions of the flagships?
If this were the case however, why would labs go through the trouble of distilling their smaller models rather than releasing quantized versions of the flagships?
With distillation you're training new model, so size of it is arbitrary, say 1T -> 20B (50x) reduction which also can be quantized. AFAIK distillation is also simply faster/cheaper than training from scratch.
"Never attribute to malice that which is adequately explained by stupidity."
Yes, I'm calling labs that don't distill smaller sized models stupid for not doing so.