It seems to me less of an issue that LLMs produce code that aren't compliant with language specs, but rather that LLMs produce code that do not meet the stated requirements, or that they break from earlier stated requirements as new requirements are requested.
And of course, code not meeting requirements is not a problem native to LLMs, but to all code production agents everywhere (including humans).
This nails one of the biggest issues that we're trying to help Mito users with -- code that meets __changing__ requirements. ie: the input data column headers change.
The best approach we have right now is giving users flexibility to parametrize their script (ie: tell the script the right column header or reference columns by position instead of name)
It turns out that teams that have moved their data to Snowflake tend to avoid these issues primarily because the schema of their data changes much less frequently.