Hacker News Reader

My point is that LLMs can do both, the same way humans can. We don't learn or recall reversals automatically either - we memorize them separately, or run an inference step (which, done often enough, leads to memorization). Reversals aren't free - not for LLMs, and not for our own minds.

View on HN

Yes the difference is that LLMs learn statically: they train and that’s it. Humans dynamically: they revisit learned facts and synthesize new ones. If you force an LLM-like training process in a human, by doing rote memorization without any exercises, then you get LLM-like results in that human.

This suggests that you may need to train an LLM on its own output. Goes against the conventional wisdom!

I think this “wisdom” is not really agreed upon, and in fact I’ve seen a few works successfully leverage synthetic data (including a cool one from Sergey Levine et al recently).

Reply on news.ycombinator.com