We've had this for over 1 year in Lamini - https://lamini-ai.github.io/inference/json_output/
Works with any open LLM, including Llama 3.1
Works with any open LLM, including Llama 3.1
https://github.com/ggerganov/llama.cpp/tree/master/grammars
Supports an EBNF-like syntax, as well as JSON-Schema.