The limitations of LLMs that solely rely on the transformer model are known: these are purely probabilistic systems, which predict the most likely next token. No reasoning ability whatsoever.
When combining the LLM with the symbolic engine, it seems that the process is much closer to the way humans would reason to solve a problem.