are you using MTP with concurrent requests? I think there could be an issue with KV cache I had with llama.cpp that randomly threw in characters from other thread's KV cache...