Hacker News Reader

Comment by unsupp0rted | Hacker News Reader

Isn't "emergent properties" another way to say "we're not very good at understanding the capabilities of complex systems"?

View on HN

I've always understood it more to mean, "phenomena that happen due to the interactions of a system's parts without being explicitly encoded into their individual behavior." Fractal patterns in nature are a great example of emergent phenomena. A single water molecule contains no explicit plan for how to get together with its buddies and make spiky hexagon shapes when they get cold.

And I've always understood talking about emergence as if it were some sort of quasi-magical and unprecedented new feature of LLMs to mean, "I don't have a deep understanding of how machine learning works." Emergent behavior is the entire point of artificial neural networks, from the latest SOTA foundation model all the way back to the very first tiny little multilayer perceptron.

Emergence in the context of LLMs is really just us learning that "hey, you don't actually need intelligence to do <task>, turns out it can be done using a good enough next token predictor. We're basically learning what intelligence isn't as we see some of the things these models can do.

I always understood this to be the initial framing, e.g. in the Language Models are Few Shot Learners paper but then it got flipped around.

Or maybe you need intelligence to be a good enough next token predictor. Maybe the thing that “just” predicts the next token can be called “intelligence”.

Maybe?

Mostly I just think that "Intelligence" and "AI" go together like "life, the universe and everything" and "42".

The challenge there would be showing that humans have this thing called intelligence. You yourself are just outputting ephemeral actions that rise out of your subconscious. We have no idea what that system feeding our output looks like (except it's some kind of organic neural net) and hence there isn't really a basis for discriminating what is and isn't intelligent besides "if it solves problems, it has some degree of intelligence"

To return an old but still good analogy ...

If you want to understand how birds fly, the fact that planes also fly is near useless. While a few common aerodynamic principles apply, both types of flight are so different from each other that you do not learn very much about one from the other.

On the other hand, if your goal is just "humans moving through the air for extended distances", it doesn't matter at all that airplanes do not fly the way birds do.

And then, on the generated third hand, if you need the kind of tight quarters maneuverability that birds can do in forests and other tangled spaces, then the way our current airplanes fly is of little to no use at all, and you're going to need a very different sort of technology than the one used in current aircraft.

And on the accidentally generated fourth hand, if your goal is "moving very large mass over very long distance", the the mechanisms of bird flight are likely to be of little utility.

The fact that two different systems can be described in a similar way (e.g. "flying") doesn't by itself tell you that they are working in remotely the same way or capable of the same sorts of things.

doesn't by itself tell you that they are working in remotely the same way or capable of the same sorts of things.

I believe any intelligence that reaches 'human level' should be capable of nearly the same things with tool use, the fact it accomplishes the goal in a different way doesn't matter because the systems behavior is generalized. Hence the term (artificial) general intelligence. Two different general intelligences built on different architectures should be able to converge on similar solutions (for example solutions based on lowest energy states) because they are operating in the same physical realm.

An AGI and an HGI should be able to have convergent solutions for fast air travel, ornithopters, and drones.

There is no "human level" because we don't even understand what we mean by "human level". We don't know what metrics to use, we don't even know what to measure.

> Two different general intelligences built on different architectures should be able to converge on similar solutions (for example solutions based on lowest energy states) because they are operating in the same physical realm.

Lots of things connected to intelligence do not operate (much) in any physical realm.

Also, you've really missed the point of the analogy. It's not a question of whether AGI would pick the same solution for fast air travel as HGI. It is that there are least two solutions to the challenge of moving things through the air in a controlled way, and they don't really work in the same way at all. Consequently, we should be ready for the possibility that there is more than one way to do the things LLMs (and to some degree) humans do with text/language, and that they may not be related to each very much. This is a counter to the claim that "since LLMs get so close to human language behavior, it seems quite likely human language behavior arises from a system like an LLM".

I think that many birds gets too sensitive when discussing what "flight" means, heh

A better bird analogy would be if we didn't understand at all how flight worked, and then started throwing rocks and had pseudo-intellectuals saying "how do we know that isn't all that flight is, we've clearly invented artificial flight".

> "we've clearly invented artificial flight"

Scaling laws shows that the harder we throw the rock the further we fly, we just have to throw them hard enough and we have invented flying rocks!

And for the naysayers out there, lemme throw this rock at your head and then tell me it isn't real!

If your goal is to get to stable orbit, even never having learned to fly, then the brute force approach works too

If we use some metric as proxy for intelligence, emergence simply means a non-linear sudden change in that metric?

Or more generally "fitting a model to data".

Not quite. Complex systems can exhibit macroscopic properties not evident at microscopic scales. For example, birds self organize into flocks, an emergent phenomenon, visible to the untrained eye. Our understanding of how it happens does not change the fact that it does.

There is a field of study for this called statistical mechanics.

https://ganguli-gang.stanford.edu/pdf/20.StatMechDeep.pdf

Very interesting crossover!