No magic needed, the quantization found the expert which deals in java and deleted it, so the model overall became better.
lol. JDQ -- Java Delete Quantization it's the new thing!
Nice
You could… actually do that. Same technique as abliteration. Run prompts until it can’t write Java lol. I bet it becomes worse at programming in general though :(
I waiting for the day when memory would stop being the bottleneck, when we will load specific parts of the thing on-demand and actually know which part does what.
Ah, I see you've found the AbstractExpertRemovalFactoryFactory!