Hacker News Reader

After using frontier models it’s hard to understand why anyone would think this is the path to AGI. Self improving models will likely have limited ability and returns. There may be breakthroughs that enable more general self improvement but the current state of frontier models isn’t that.

View on HN

Thing is, it depends on whether llms + reinforcement can self-improve in principle. Learned recently that cognitive scientists, before the transformer & llms, were studying the possibility that thinking and learning might be based on some kind of prediction, i.e. something similar to token prediction, and I quite suddenly became less skeptical about the possibilities of llms. (Some will say I’m late to the party of course.) But if knowledge to date has been accumulated in a process quite like “chain of thought” in llms, then I don’t see any reason that computers won’t self-improve in the near future.

At a high level, we're still at the stage of AI development where we're taking cues from nature.

Take the most recent qwen and deepseek models with offloadable n-grams, which function (both in name and vaguely in capability) like human memory "engrams".

Huh, TIL "engram" is not just an alternate spelling of "n-gram".

Reply on news.ycombinator.com