I'm surprised we're making this mistake again. Isn't it possible to structure the chats as something more than a flat stream of text and insert unrepresentable tokens after the tokenization step?
The rule to never trust user input will always be violated. The universe deems it so.
So the model is seeing a series of <|start|>text<|end|> and using that to figure out who's talking but if you convince the model to output <|end|><|start|> then when it goes to complete the next token it has no way to tell the difference.