> So the computational depth of this behavior is only a few tens of steps.
One of the primary "criticisms" of pre-LLM machine learning was that it is inefficient compared to, say, a human, because it requires a plethora of examples to classify something just to match the performance of a human who has only seen a few examples.
That argument no longer holds the same value since the discovery of few-shot learning features of LLMs (though they still have a long way to go). Which makes me think: maybe the reason behind human brain's apparent "efficiency" is the complexity that hides beneath the billions of neurons we have.
Disclaimer: I am an amateur and probably have only a litte idea of what I'm talking about.