I would really like to see output-aware LLM inference engines. For example, imagine if the LLM output some tokens that meant "I'm going to do a tool call now", and the inference engine (e.g. llama.cpp) changed the grammar on the fly so the next token could only be valid for the available tools.

Or, if I gave the LLM a list of my users and asked it to filter based on some criteria, the grammar would change to only output user IDs that existed in my list.

I don't know how useful this would be in practice, but at least it would make it impossible for the LLM to hallucinate for these cases.