That article explains that the quirk was introduced during supervised fine-tuning.

> We unknowingly gave particularly high rewards for metaphors with creatures

It was the human feedback that caused the bias, not a change in training data.