Real reasoning, which can be used to predict outcomes in novel situations, is based on multi-step what-if prediction, perhaps coupled with actual experimentation, and requires things like long-term (task duration) attention, (task duration) working memory, online learning (unless you want to have to figure everything out from scratch everytime you re-encounter it), perhaps (depending on the problem) innate curiosity to explore potential solutions, etc. LLMs are architecturally missing all of the above.

What you might call "fake" reasoning, or memorized reasoning, only works in situations similar to what an LLM was exposed to in it's training set (e.g. during a post-training step intended to embue better reasoning), and is just recalling reasoning steps (reflected in word sequences) that it has seen in the training set in similar circumstances.

The difference between the two is that real reasoning will work for any problem, while fake/recall reasoning only works for situations it saw in the training set. Relying on fake reasoning makes the model very "brittle" - it may seem intelligent in some/many situations where it can rely on recall, but then "unexpectedly" behave in some dumb way when faced with a novel problem. You can see an example of this with the "farmer crossing river with hen and corn" type problem, where the models get it right if problem is similar enough to what it was trained on, but can devolve into nonsense like crossing back and forth multiple times unnecessarily (which has the surface form of a solution) if the problem is made a bit less familiar.