memory, internal iteration and scratchpads can and have been added. Foundation models don't include this yet, but it's not unexplored territory. Online learning is more difficult but I'd argue that various fine-tuning and control approaches are slowly working towards that.

So what is a brain other than a very complex non-linear function which takes inputs, iterates on them, integrates them with memory and eventually generates output?

> is that it doesn't truly understand anything at all, but is just instead (in simple terms) a large matrix of inputs to outputs

The first step to defeating the tiger is to realize that it cannot hurt you, for it is only made of simple atoms.