Reminds me a bit about the SolidGoldMagikarp: https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldm... . Even though the SolidGoldMagikarp was clearly a bug in the tokenizer.
There was no problem with the token per se, but the fact it was like a strange attractor in multidimensional space, disconnected from any useful information.
When the LLM was induced to use it in its output, the next predicted token would be random gibberish.