It's basically guidelines to adhere in the output. Ex. Your prompt asks for a summary, the LLM won't necessarily just spit the summary, there might be phrases like here's the summary - then the actual summary or even basic JSON key/values can get messed up. So grammar allows for defining the expected output in a variety of ways, some via Pydantic class definitions, Microsoft's Guidance, Outlines along with llama.cpp's grammar are attempts at making structured output reliable.

Most of langchain is basically a specific prompt with exception handling for missing fields.

You can give it a formal definition for valid JSON, and it will only generate output that matches.