This is just nonsensical changes and slurs, but particularly degenerate input data can cause big issues in training:

https://x.com/gabriberton/status/2051873677998956851