Byte-Level Tokenizers Unavoidably Enable LLMs to Generate Ill-Formed UTF-8

You can enforce grammar. Llama.cpp can do that, without any need to change models.