Hacker News Reader

Comment by ux-app | Hacker News Reader

>However it is still just pattern matching

the visual cortex in your brain is also "just a pattern matching" system. guess it's not very impressive by your standard.

This[1] isn't my example (it's from another HN user), but if you work as a programmer and you're not absolutely jaw on the floor astonished by this example then I don't know what to say.

Explaining[2] the emergent behaviour is literally cutting edge research. Hand waving this behaviour away as just "probabilistically generating a likely next word" is ignorant.

It's amazing in similar ways to Conway's Game of Life.

[1] https://imgur.com/HOEnxYb

[2]https://ai.googleblog.com/2022/11/characterizing-emergent-ph...

View on HN

I'm arguing against the notion that these LLMs exhibit "emergent behaviour" as you stated. I don't believe they do, as the term is commonly understood. Emergent behavior usually implies the exhibition of some kind of complexity from a fundamentally simple system. But these LLMs are not fundamentally simple, when considered together with the vast corpus of training data to which they are inextricably linked.

The emergent behavior of Conway's Game of Life arises purely out of the simple rules upon which the simulation proceeds -- a fundamental difference.

did you read the article?

emergent behavior in this context is defined as: "emergent abilities, which we define as abilities that are not present in small models but are present in larger models"

>The emergent behavior of Conway's Game of Life arises purely out of the simple rules upon which the simulation proceeds -- a fundamental difference.

this is a meaningless distinction.

> emergent behavior in this context is defined as: "emergent abilities, which we define as abilities that are not present in small models but are present in larger models"

Then I don't know why you brought up Game of Life because it obviously has nothing to do with this alternative definition of emergent behavior.

> this is a meaningless distinction.

It's meaningful with respect to the claim that LLMs exhibit emergent behavior in the same way in which Game of Life does.

>It's meaningful with respect to the claim that LLMs exhibit emergent behavior in the same way in which Game of Life does.

I said it's amazing in __similar__ ways to Conway's Game of Life.

i.e. a system which behaves in unexpected ways (emergent abilities) and is greater than the sum of its parts.

A propos of [1]

1. Item 3: The ocean is full of floating objects, and it would be hard to see the duck among them? 2. Item 2: is structured as non sequitur, takes a long time because there are many hazards?

I am impressed that you find it impressive. It is plausible-sounding, and I find that disturbing, but it is not useful (and the text prediction paradigm seems a dead end in terms of formulating anything more than plausible sounding)

> the visual cortex in your brain is also "just a pattern matching" system

I can think and a pattern matching system can't.

> you're not absolutely jaw on the floor

Not at all. Autocomplete will autocomplete.

What is thinking?

You are asking the right question.

Reply on news.ycombinator.com