This behavior can, in turn, be transferred via distillation. But, evidently, financial domain wasn't entangled enough with the censorship behaviors for them to bleed through, in this case.
This behavior can, in turn, be transferred via distillation. But, evidently, financial domain wasn't entangled enough with the censorship behaviors for them to bleed through, in this case.
There is just too little overlap in the transferred knowledge.
[0]: https://github.com/CTGT-Inc/lineage-eval/blob/main/data/benc...
[1]: https://github.com/CTGT-Inc/lineage-eval/blob/main/data/benc...
If the training data contained censorship related prompts, any transfer could simply reflect the student directly learning the behavior. Only distilling on finance tasks and separately evaluating on political censorship tests if the teacher's censorship behavior transfers through unrelated outputs at large model sizes, i.e. subliminal learning (https://arxiv.org/abs/2507.14805).