Programming languages are artificial languages. LLM are able to synthesize human languages with almost perfect grammatical quality, they are in fact, very unlikely to make obvious syntactic errors on programming languages.

Also, syntax level information are local or short sighted, it is called context-free grammar for a reason. My own observation with playing with those coding LLMs all day, is that they most likely had acquired the grammar themselves implicitly. Providing explicit regularization by enforcing grammar, is going to provide at best modest benefits, and that is dependent on good that parser is written, in many cases, it is not a given.