Comment by naasking | Hacker News Reader

Comment by naasking | Hacker News Reader

Parent Full thread

I think "scale-dependent capability" is a much more precise term for what they're describing, but I'm not sure that that term doesn't fall under the general umbrella of emergent properties. The opening paragraphs of emergent properties in philosophy [1] cites a number of examples that I would argue are comparable to LLM suddenly becoming able to do arithmetic past a certain scale.

> In philosophy and esp. the philosophy of science, emergence is a relation between a whole and its parts such that a property of the whole does not obtain just in virtue of properties of its parts taken in isolation.

This is not a settled definition. I think everyone can agree that this applies epistemologically, where studying the parts in isolation cannot always yield sufficient information to predict macroscopic properties, but to claim that all properties of the whole do not reduce to the properties of its parts is controversial.

For instance, it seems unlikely that we would have predicted H2O's dipole moment and the phenomenon of surface tension just from studying hydrogen atoms and oxygen atoms in isolation, but it would be incorrect to say that surface tension is not the result of the properties of hydrogen and oxygen in isolation. We simply cannot discover all the relevant properties without studying them together.

Edit: to clarify, H2O's dipole moment seems obvious in hindsight when we have a good model for what's going on, and analogously, LLM's ability to do arithmetic seems obvious as a scale-dependent property in hindsight, but that doesn't mean it was obvious that this would happen before it was created.

[1] https://plato.stanford.edu/entries/properties-emergent/

Well it's scale dependence is an illusion.

It's uniformly able to model Q->SingleAnswer problems, and uniformly able to model Q-> ManyAnswer problems.

Basic arithmetic is the former kind, and it becomes useful to people at large scales, ie., accurate on basic arithmetic.

This dual condition of "useful-to-people" is the thing introducing the illusion, since it changes depending on what we're modelling. The system isnt acquring any new property.

Consider a researcher putting a book on a thin ice-sheet, and then putting a car on it. Here, they're concluding the ice has different properties in each case -- but it doesnt.

> This dual condition of "useful-to-people" is the thing introducing the illusion, since it changes depending on what we're modelling. The system isnt acquring any new property.

I have a couple of possible responses to this, but maybe the most obvious is that I'm not sure why "useful to people" can't qualify as a new property.

For instance, a system that suddenly becomes useful to people can be transformative to society, which can lead to new emergent social or economic properties at the societal scale. To conclude that "useful to people" is not a meaningful property, aren't you basically implying that something that suddenly becomes useful cannot even in principle lead to new societal scale emergent properties? That seems dubious. Edit: or you're implying that emergent properties are not reducible to interactions between constituent properties, which also seems dubious.

For a concrete example, the internet probably falls into this category. It has transformed society and led to new emergent properties at societal scales, but computers didn't suddenly acquire any new computational properties, or new properties to manipulate bits. Only the scale of their deployment changed, and that scaling was itself useful to people, and this led to new societal properties. That arguably can't happen unless "useful to people" is itself a meaningful property, no?

> Consider a researcher putting a book on a thin ice-sheet, and then putting a car on it. Here, they're concluding the ice has different properties in each case -- but it doesnt.

This is just a linguistic shell game with the meaning of the word "property". You could just as easily say the difference between the mind of a human and a monkey is a matter of degree, and therefore going from one to the other does not gain any novel "property".

It should be obvious that the degree of a property can fundamentally change its nature, and that there is no hard distinction between "properties" and degrees of things. The difference between a tickle and a gunshot are matters of "degree", but that fact is of near zero semantic utility.

Emergence is about intrinsic properties, observer-independent properties.

If emergence were about observer-relative properties it would be a meaningless term. My shoe gets an "emergent property" to hold my door open when I put it in a door way.

This is mumbojumbo.

Systems acquiring observer-relative "properties" are all well and good, but the claim here is a much stronger one.

This gross misuse of language amounts to saying that "models with enough parameters to accurately approximate a function" have "emergent properties" that "models without enough parameters" do not have.

This is a deeply silly way to describe the conditions under which one function can approximate another, and the rate at which that aproximation converges to something useful.

Let's make this concrete. What in your mind is a specific example of a concrete system with an emergent property, then?

Liquidity is not a property of h2o molecules but it is of water. Liquidity isn't of water without a container/pressure environment.

The trajectories of particles of air are underdetermined by their own intrinsic properties so a tornado cannot be reduced to some mere aggregate of them.

Emergence is an ontological relationship between properties of objects --- it isn't a mathematical property of a data distribution nor of an approximation function.

The very use of the term has crated all this confusion right here.

Would anyone who thought NNs were showing emergence be also content to find out that the reason for this 'emergence' was just that in the case of so-called 'emergence' our expectations of performance were changing?

Do we call it 'emergence' when we replace estimating data using mean() to estimating with ax+b ?

There's definitely an illusion here, but one quite easy to spot if you weren't in the game of peddling illusions.

Two observations,

There is utility and need for some convention of feature description, for the case of the external behavior of the system being correct, for some domain, despite lack of specific training. It is reasonable for users of such systems to say that the internal states and representation don't matter, so long as the behavior is correct; and in cases like these we will benefit from some consensus on how to talk about such things. Fine with me if some new term is applied.

More of interest to me though is that it is not at all clear to me that genuine emergence is not possible through scaling (independent of whether it is in any given existing LLM). Because the optimal (most compact) correct representation for a lot of e.g. language output, is exactly that which benefits from abstraction.

What reason is there to believe that the abstractions derived at higher levels (of the network generally but not necessarily, depends on the architecture) do not encode non-linear problem spaces in the world, which are "real" emergence?

I.e. if the way some network learns arithmetic is to settle on an internal weighting that performs computation, rather than "memorizing assertions", me, I would call that "emergent."

But I'm happy to use some other term should one, er, emerge.

> Liquidity is not a property of h2o molecules but it is of water

The ability to speak English is not a property of floating points, but it is of certain, very specific large tensors of them. What's the difference?

> Emergence is an ontological relationship between properties of objects --- it isn't a mathematical property of a data distribution nor of an approximation function.

I don't see a hard distinction between ontological relationships and data distributions. All information is fundamentally statistical. Our access to ontology is forever and always mediated by "data distributions".

One could, of course posit that there are fundamental, non-statistical ontological things out there. However, the liquidity of water being an ontological relationship while the English-speaking of GPT not being so is merely a hypothesis, not an objective fact of the universe, at least not as far as I can tell.

> What in your mind is a specific example of a concrete system with an emergent property, then?

Not sure if you're joking, but the brain (and consciousness/mind as an emergent phenomenon) is the classic example. Even better, it is what's causing the fundamental problems in this very conversation, due to the inconsistent manner in which it translates terms into meaning (ie: "emergent", "is"), typically not realizing[1] that meaning and reality are similar to the speed of light in that their appearances[2] vary depending upon the frame of reference of the observer.

I am fairly optimistic that AI is going to "force humanity's cultural hands" such that we will eventually have to grapple with this long known but rarely discussed phenomenon. Though, I anticipate censorship, propaganda, and consciousness will play heavy roles in such discussions and screw everything up, as is usually the case with fundamentally important ideas.

[1] During realtime cognition, regardless of whether substantial abstract knowledge is in the person's possession.

[2] And maybe even the things themselves, depending on how (or from where) you look at it - I am still undecided on this.

> Emergence is about intrinsic properties, observer-independent properties. If emergence were about observer-relative properties it would be a meaningless term.

I'm going to address this in case it was also intended to reply to my other comment about "useful to people" possibly being a property.

"Useful to people" would be an observer-independent property, if it's a property at all. An alien species analyzing humanity would come to the same conclusions as humans about whether some system, like the internet, was useful to people. This would be evident by whether the use of that system spread.

As for whether it's "intrinsic", I'm not sure how you're applying this. As you said in a later comment, "Liquidity isn't [a property] of water without a container/pressure environment". In other words, liquidity isn't an intrinsic property of H2O. Moreover, the only reason we identify and created a label for "liquidity" is because it's useful to people, which is the very criterion that you're claiming should not be applied to describe some surprising scaling behaviour of LLMs.

I just don't think you've made the distinction you're attempting to make clear, because there is parity of reasoning between the allegedly emergent properties you describe and those in the article.

Reply on news.ycombinator.com