Source code is, for the GPL, the “preferred form for modification”. Given the outrageous cost of training from scratch, plus the ability to fine tune models... I don't think it's completely ridiculous.
Edit: plus, the industry acts like models aren't derivative works of the training data. If true the source license is less important than the model!