Oh, interesting. "!" is token id 0 in the Qwen tokenizer; I wonder if there's some tokenizer shenanigans either in inference or training that end up causing this specific behavior.

Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?

Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error.