I think this is pretty insightful actually, the fact that even something as basic as predicting more than one token is really in effect the result of an outside harness. More complex things like memory, where people implement them using RAGs or vector databases, I would definitely classify as poking and honestly seem like a hack to me. And this is what I've been thinking for a while: it's hard to reconcile the idea that we can get "AGI" (however you define it) with such a system that is completely stateless. Yet, despite this statelessness, they can go ahead and solve Millenium Prize problems (with sufficient compute). It's hard to reconcile.