I can say that this really works great, I'm a heavy user of the unsloth dyanmic quants. I run DeepSeek v3/r1 in Q3, and ernie-300b and KimiK2 in Q3 too. Amazing performance. I run Qwen3-235b in both Q4 and Q8 and can barely tell the difference so much so that I just keep Q4 since it's twice as fast.
Thanks for using them! :)
What hardware do you use, out of curiosity?
In the current era of MoE models, the system RAM memory bandwidth determines your speed more than the GPU does.