Sounds like most of this is simply taking shortcuts instead of properly parsing[0].
0: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
0: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
Unfortunately that, er, opportunistic shortcut is an essential behavior of modern LLMs, and everybody keeps building around it hoping the root problem will be fixed by some silver-bullet further down the line.