Hacker News Reader

Comment by dleary | Hacker News Reader

> It literally is "in our training data similar concepts were clustered with some other similar concepts, and manipulating these clusters lead to different outcomes".

Recognizing concepts, grouping and manipulating similar concepts together, is what “abstraction” is. It's the fundamental essence of both "building a world model" and "thinking".

> Nothing in the link you provided is even close to "neurons, model of the world, thinking" etc.

I really have no idea how to address your argument. It’s like you’re saying,

“Nothing you have provided is even close to a model of the world or thinking. Instead, the LLM is merely building a very basic model of the world and performing very basic reasoning”.

View on HN

A lot of people have been bamboozled by the word 'neuron' and extrapolated that as a category error. It's metaphorical use in compsci is as close to a physical neuron as being good is to gold. Put another way, a drawing of a snake will not bite you.

> Recognizing concepts, grouping and manipulating similar concepts together,

Once again, it does none of those things. The training dataset has those concepts grouped together. The model recognizes nothing, and groups nothing

> I really have no idea how to address your argument. It’s like you’re saying,

No. I'm literally saying: there's literally nothing to support your belief that there's anything resembling understanding of the world, having a world model, neurons, thinking, or reasoning in LLMs.

> there's literally nothing to support your belief that there's anything resembling understanding of the world, having a world model, neurons, thinking, or reasoning in LLMs.

The link mentions "a feature that triggers on the Golden Gate Bridge".

As a test case, I just drew this terrible doodle of the Golden Gate Bridge in MS paint: https://imgur.com/a/1TJ68JU

I saved the file as "a.png", opened the chatgpt website, started a new chat, uploaded the file, and entered, "what is this?"

It had a couple of paragraphs saying it looked like a suspension bridge. I said "which bridge". It had some more saying it was probably the GGB, based on two particular pieces of evidence, which it explained.

https://imgur.com/a/sWwWlxO

> The model recognizes nothing, and groups nothing

Then how do you explain the interaction I had with chatgpt just now? It sure looks to me like it recognized the GGB from my doodle.

You're Clever Hans-ing yourself into thinking there's more going on than there is.

Machine learning models can do this and have been for a long time. The only thing different here is there's some generated text to go along with it with the "reasoning" entirely made up ex post facto

> Then how do you explain the interaction I had with chatgpt just now?

Predominantly English-language data set with one of the most famous suspension bridges in the world?

How can anyone explain the clustering of data on that? Surely it's the model of the world, and thinking, and neurons.

What happens if you type "most famous suspension bridges in the world" into Google and click the first ten or so links? It couldn't be literally the same data? https://imgur.com/a/tJ29rEC

Reply on news.ycombinator.com