Research done by Prof. Subbarao on reasoning & planning with LLM seems to point in the negative.

https://www.youtube.com/watch?v=uTXXYi75QCU

He has lots of good threads distilling his research. https://twitter.com/rao2z