And there comes the fact LLMs are actually terrible at coding beyond advanced regurgitation, precisely because their single-pass architecture does not give them Turing completeness to model a Turing machine, which may also be the reason they model the full chain of cause and effect to formulate a coherent plan...

There are people working on that! Google for LLM + PDDL.

There are big problems with hallucinations because LLMs are not smart enough to know when they're starting to make mistakes.

But there's lots of work in this area, and generally in different ways to nail neural and symbolic systems together.