Ideally a recursive execution would also be a pure function - maybe a better way to put it about current LLMs is that they are a single mathematical expression being built up from a fix number of nodes and only addition and multiplication.
“You learn best from your mistakes” comes to mind here.
And not just shallow feedback loops but bigger, more profound ones eg “I didn’t go to work for a month and so I was fired and and now I have no money and it will now be hard to be hired again”.