Oh I would always use Q2_K_XL :) It uses our dynamic methodology to quantize certain layers in different bits ie 2, 3, 4, 5, 6, 8 bits - the more important the layer is, the higher the bitrate
How do you determine the importance of a layer in this case?
Afaik they have a test bench that they use and take the activation data from that.
Yes we have around 1 to 3 million tokens of high quality self verified data that we use to calibrate models!