Hacker News Reader

Comment by mo_42 | Hacker News Reader

> LLMs aren’t people, and emergent properties are being over extended.

LLMs are trained on human texts. And if they're trained well these models might start to simulate parts of a human brain. Might be that this is the simplest way to produce texts that satisfy the objective function.

If this would be the case, and we give LLMs some emotional shading, maybe these would simulate parts of a human brain with this emotion and hence produce better answers (given that in the training data, emotional samples have better quality).

EDIT: added "parts of" before human brain.

View on HN

It won't ever simulate the human brain. It may simulate human cultural knowledge, or emotions, but only as far as they are encoded in the current millennium's written knowledge.

The human brain doesn't even have the concept of written language, that's all culturally learned knowledge.

Emulate the brain? No. Simulate the brain? Yes.

It’s not the brain, it’s the mind or what Plato would have called the Nous.

How would you test whether some arbitrary thing is simulating the human brain?

If you have no answer, I put it to you that your assertion is of the no-true-Scotsman type -- that is, unfalsifiable.

Its not a no-true-Scotsman type argument in a world where the existence of Scotland itself is purely hypothetical, and the arguments for the possibility of its existence in the form of a working bagpipe demonstrator are also an illustration that you don't need to be Scottish to play bagpipes.

The impossibility of testing whether the brain is adequately simulated (since a pretty basic LLM that makes no attempt to simulate the functions of a human mind yields human-like text i/o) is a point in favour of it being unlikely that we could design a fully functioning simulation of a human mind out of arbitrary material(1), since the inadequacy of testing is an impediment to actually building it.

(1) it's apparently possible for us to create human minds out of bits of humans and a process called pregnancy

I had to apply some serious analogy-algebra, but I think your first paragraph is equating Scotland to intelligence -- rather than to the human brain specifically, which is the argument GP and I are having. The human brain is not hypothetical, and despite it being outside the realms of today's technology, it's likely that one day it will be possible to simulate one. At that time it will be possible to gauge whether today's LLMs are accurate approximations.

No, my first paragraph sidesteps messy debate about what intelligence is and focuses on the simple fact a human brain simulation exists purely in the realms of the hypothetical.

Its not the no true Scotsman fallacy unless the Scotsman actually exists.

What is the "working bagpipe demonstrator" in your analogy? And what is the meaning of "you don't need to be Scottish"?

An LLM. Not a brain simulation, but mimics the text outputs of brain activity quite well just by parsing text. Turns out that like getting an Englishman to play bagpipes, you can get an LLM to write as if it's angry or drunk or horny (A corollary of a sufficiently large text learning model generating convincingly angry outputs based on pure word association is that you can't trust an attempt to build a long running process with adrenaline and cortisol analogues has adequately simulated emotional state just because the communications module writes convincingly angry responses)

So it's literally the inverse of "no true Scotsman". We don't have Scotsmen not doing things that all "true Scotsmen" are supposed to do, we have "definitely not Scotsmen" passing benchmarks for Scottishness (in a world in which Scotland itself is only an aspiration)

> How would you test whether some arbitrary thing is simulating the human brain?

In my first comment, I didn't claim that it's simulating the entire human brain but aspects of it like language and memory.

We can test such aspects by throwing text at the model and comparing it to responses of humans. A simulation doesn't need to have the same abstractions. What counts is solely the quality of the response. If the responses are sufficiently close to those of humans, I'd say it's simulating a human brain.

First note the difference between "it's simulating the human brain" and "it's behaving in an intelligent way". Only one of the two requires tracking of blood sugar levels (to explain fatigue) and controlling motor neurons.

We happen to know how LLMs are trained, and from the loss function it's pretty clear that blood sugar will not magically enter the equation, nor even motor neurons, except as abstract knowledge that the model can reason about.

A LLM trained on western texts will, best-case, simulate a western abstract reasoning process, which usually puts more focus on independent/isolated explanations compared to other cultures. From that training data it will not spontaneously start simulating a generic human brain that was separated from both its cultural knowledge and its spinal cord.

At least in the same sense that the Mandelbrot fractal does not contain a picture of the maxwell equations. You may actually find that picture, but by looking hard enough in a complex enough system, you can find just about anything else you want.

An LLM doesn't even have the concept of written language. That's all culturally learned knowledge.

We can with reasonable certainty say that there are still significant differences between LLMs and human brains, but this notion that we can say with any certainty that similar structures won't form as a side effect of sufficient training is pure fiction.

Reply on news.ycombinator.com