Comment by opportune | Hacker News Reader

Comment by opportune | Hacker News Reader

Parent Full thread

I’m sorry you find it obnoxious but emergent phenomena are everywhere in math and science and as annoying as it is to you, it also happens with AI.

The quest for more generalized models boils down to studying emergent behavior because we could never prescriptively define all the parameters/behaviors/requirements necessary for such a complex outcome. We don’t even understand how the relatively easily observed interactions between neurons in our own brains result in emergent intelligence.

What’s so impressive about LLMs is they understand the semantics of some concepts so well that they can consistently produce higher quality outputs for tasks like “explain this complicated concept with a nursery song from the perspective of a pirate” than humans could, with approximately no instances of that task in their training data. That is emergent behavior and it’s a pretty big deal.

I agree that emergent behaviors are real, and important.

I am skeptical, although not completely unconvinced, that LLMs like GPT are going to produce truly emergent phenomena, such as true first-principles logical reasoning. The limitations of the underlying transformer architecture itself are, in my opinion, the problem. The first problem is that the embedding space of the transformer needs to grow much, much larger, and it's already huge. This matters because you need to model the order of neurons in the brain. The second problem is that you're never going to train an LLM (as they're designed today) that is going to produce a truly good 'emergent-phenomena' answer without multiple network traversals. This is because the human mind constantly and autonomously refines its thoughts.

Perhaps a good counter-argument is that emergent phenomena are fundamentally a space-time domain concept.

I am aware that things like Conway's Game of Life are a fantastic counterargument to my "the transformer architecture doesn't support it" argument. But I agree that the definition of "emergent behavior" when it comes to machine learning is too easily corrupted to be novel rather than rigorous.

Reply on news.ycombinator.com