Hacker News Reader

Comment by cs702 | Hacker News Reader

I view this as compelling evidence that current models are more than "stochastic parrots," because as the OP shows, they are learning to model the world in ways that are similar (up to a linear transformation) to those exhibited by the human brain. The OP's findings, in short:

* A linear transformation of a speech encoder's embeddings closely aligns them with patterns of neural activity in the brain's speech areas in response to the same speech sample.

* A linear transformation of a language decoder's embeddings closely aligns them with patterns of neural activity in the brain's language areas in response to the same language sample.

View on HN

Yeah, I have always firmly maintained that there is less fundamental difference between LLMs and human brains than most people seems to assume.

Going a bit further, I'll speculate that the actions made by a human brain are simply a function of the "input" from our ~5 senses combined with our memory (obviously there are complications such as spinal reflexes, but I don't think those affect my main point). Neural nets are universal function approximators, so can't a sufficiently large neural net approximate a full human brain? In that case, is there any merit to saying that a human "understands" something in a way that a neural net doesn't? There's obviously a huge gap between the two right now, but I don't see any fundamental difference besides "consciousness" which is not well defined to begin with.

The UAT is a pretty weak result in practice. A lot of systems have the same property, and most of them are pretty poor approximators in practice. It may very well be that no reasonable amount of computing power allows approximating the "function of consciousness". Plus, if you're a certain kind of dualist the entire idea of a compact, smooth "consciousness" function may be something you reject philosophically.

I agree there are issues with the UAT, but I feel like my conclusion is still valid: a neural net, given the memories and senses that a humans has, is capable of approximating a human's response accurately enough to be indistinguishable from another human, at least to another human.

I philosophically reject the notion that consciousness is an important factor here. The question of whether or not you have a consciousness doesn't affect what I take away from this conversation, and similarly the question of whether an AI has a consciousness doesn't affect what I take away from my actions with it. If the (non-)existence of others' consciousnesses doesn't materially affect my life—and we assume that it's a fundamentally unanswerable question—why should I care other than curiosity?

>a neural net, given the memories and senses that a humans has, is capable of approximating a human's response accurately enough to be indistinguishable from another human, at least to another human.

That doesn't remotely follow from the UAT and is also almost certainly false.

I am not saying that follows from the formal UAT, I'm saying that in general, if we assume that the human brain is simply producing outputs in response to inputs, I have no reason to believe that a neural net couldn't be constructed to approximate the same outputs given the same inputs. And specifically, the threshold of "good enough approximation" I chose is effectively just a stronger form of the turing test. AI has arguably already passed a weaker form of the turing test (most people in the world cannot tell the difference between a human-written and an AI-written article, for example).

Ah, so there's two of us now :)

> I have always firmly maintained that there is less fundamental difference between LLMs and human brains than most people seems to assume.

What is your basis for this? Do you have any evidence or expertise in neuroscience to be able to make this claim?

> Neural nets are universal function approximators, so can't a sufficiently large neural net approximate a full human brain?

We do not understand the brain well enough to make this claim.

> but I don't see any fundamental difference besides "consciousness" which is not well defined to begin with.

Yeah besides the gaping hole in our current understanding of neuroscience, you have some good points I guess.

Computers can compute anything, with the only limitations being time, space, energy, programming, etc. So far, it seems like all of those things can be increased without bound. I see no reason to believe that there will be some fundamental limitation that prevents us from improving computers to the point that they can compute the equivalent of a human brain. Yes I agree we don't understand the human brain well enough yet, but empirically we can observe that AI agents and human agents behave in similar ways. Sure, it may not end up being a complete copy of a human brain, but if the inputs and outputs are the same, does it make a difference?

> current models are more than "stochastic parrots"

I believe the same, and also I'm willing to accept that the human brain can intentionally operate in an stochastic parrot mode.

Some people have the ability to fluently speak non-stop, completely impromptu. I wonder if it's similar to an LLM pipeline, where there'a constant stream of thoughts being generated based on very recent context, which are then passed through various output filters.

That certainly would explain how Trump can rattle on for hours and hours, despite a notable lack of high quality training data.

I do not understand what you find convincing about this that changes your mind.

We have a closed system that we designed to operate in a way that is similar to our limited understanding of how a portion of the brain works, based on how we would model that part of the brain if it had to traverse an nth-dimensional array. We have loosely observed it working in a way that could roughly be defined as similar to our limited understanding of how a portion of the brain works given that limitation that we know is not true of the human brain, with a fairly low confidence level.

Even if you put an extreme level of faith into those very subpar conclusions and take them to be rigid... That does not make it actually similar to the human brain, or any kind of brain at all.

I view this as further evidence that we less different from "stochastic parrots" than we would like to believe.

The correlations are 0.25-0.5, which is quite poor (Gaussian distribution plots with those correlations look like noise). That's before analyzing the methodology and assumptions.

Isn't correlation a wrong way of measuring statistical dependence anyway? I thought that entropy-based method like Relative Entropy did this better.

Yeah, Taleb has some pretty convincing arguments that correlation is generally bad/misleading and entropy is better.

Correlation of 0.25-0.5 being poor is very problem dependent.

For example, in difficult perceptual tasks ("can you taste which of these three biscuits is different" [one biscuit is made with slightly less sugar]), a correlation of 0.3 is commonplace and considered an appropriate amount of annotator agreement to make decisions.

Yes for certain things like statistical trading (assuming some kind of "nice" Gaussian-like distribution) where you have lots of trades and just need to be more right than wrong it's probably useful.

Not here though, where you are trying to prove a (near) equivalence.

> I view this as compelling evidence that current models are more than "stochastic parrots,"

More evidence against "stochastic parrots"

- zero shot translation, where LLMs can translate between unseen pairs of languages

- repeated sampling of responses from the same prompt - which shows diversity of expression with convergence of semantics

- reasoning models - solving problems

But my main critique is that they are better seen as pianos, not parrots. Pianos don't make music, but we do. And we play the LLMs on the keyboard like regular pianos.

Galaxies, hurricanes, whirlpools. Black Spleenwort, Barnsley Fern. Power laws every bloody where.

(My utterly uninformed knee-jerk reaction here, but even if I was a true believer I don't think I'd reach for "compelling".)

natural consequence when results are the sum of random processes.

if you add random rolls, you get a gaussian, thanks central limit theorem.

if you sum them, you get a lognormal distribution, which approximates a power law up to a cutoff

I view this as the language processing is similar but we’re not talking about thinking, just processing language.

I see this as maybe it’s not a statistical parrot, but it’s still only some kind of parrot, Maybe a sleep deprived one.

There isn't really any clear delineation between 'thinking' and 'language processing' in the brain.

That's 100% false, dogs and pigeons can obviously think, and it is childish to suppose that their thoughts are a sequence of woofs or coos. Trying to make an AI that thinks like a human without being able to think like a chimpanzee gives you reasoning LLMs that can spit out proofs in algebraic topology, yet still struggle with out-of-distribution counting problems which frogs and fish can solve.

How human brains process thoughts is non-uniform across population. There’s imagery, written language, sound, speech, tactile, etc. Not everything that you think about is readily expressible in your language. There are definitely people with and without “internal screen”, and probably few more types with/without X, where X is a set of things we’ve never talked about, either assuming everyone has it or not realizing that it’s a non-mandatory part of how you think.

That's not really what I'm saying. What I'm saying is how does the brain look when you do both ? Is there a clear difference ? There's no 'thinking mode' followed by a 'language processing mode'.

Language processing is thinking as far the brain is concerned and there's no evidence that these are 2 cleanly separated processes whether you 'think' in words or not.

That's not true, the Moro experiments show they use different capacities as do similar experiments on people who have certain severe cognitive deficiencies that don't impact language processing (e.g. the subject "Chris")

My argument is that “thinking” and “language processing” are not two sequential or clearly separated modes in the brain but deeply intertwined.

Language is a lot more than parsing syntax, whatever your thoughts are on the matter, even LLMs are clearly doing more than that. Are there any experiments where subjects had severe cognitive deficiencies and language in its full breadth or maybe i should say communication ? came out unscathed ?

The chris experiments don't seem to go into much detail in that front.

I just gave one: "Chris". Here's Chomsky describing the "Chris"-experiments ([1]) as part of a broader answer about how language is distinct from general cognition which I paraphrased above.

> That doesn't contradict the argument that “thinking” and “language processing” are not two sequential or clearly separated modes in the brain but deeply intertwined.

It's not an argument, it's an assertion, that is, in fact, contradicted by the experimental evidence I described (Moro and "Chris"). Of course they are "deeply intertwined" but because of the evidence it's probably an interface between two distinctive systems rather than one general system doing two tasks.

https://youtu.be/Rgd8BnZ2-iw?si=_UdBfG1InmpjbDPB&t=6735

Like i said, these experiments stop at a vague 'Chris can still learn languages'. No comment on actual proficiency or testing. For all i know i can't have a meaningful conversation with this guy beyond syntactically correct speech. Or maybe the best proficiency he's ever managed is still pretty poor compared to the average human. I have no idea.

There's no contradiction because i never argued/asserted the brain didn't have parts tuned for language, which is really all this experiment demonstrates.

It's irrelevant to the experiment: he could learn synthetic language with human-like grammar and could not learn synthetic languages with non-human-like grammar. Regular people could solve the non-human-like languages with difficulty. Because his language ability is much higher than his general problem solving ability it gives strong evidence that 1. human language capacity a special function, not a general purpose cognitive function and 2. it obeys a certain structure.

> There's no contradiction because i never argued/asserted the brain didn't have centers tuned for language, which is really all this experiment demonstrates.

I don't know what you are trying to say then.

>Because his language ability is much higher than his general problem solving ability

I don't see how you can say his language ability is much higher than his general problem solving ability if you don't know what proficiency of language he is capable of reaching.

When you are learning say English as a second language, there are proficiency tiers you get assigned when you get tested - A1, A2 etc

If he's learning all these languages but maxing out at A2 then his language ability is only slightly better than his general problem solving ability.

This is the point i'm trying to drive home. Maybe it's because i've been learning a second language for a couple years and so i see it more clearly but saying 'he learned x language' says absolutely nothing. People say that to mean anything from 'well he can ask for the toilet' to 'could be mistaken for a native'.

>I don't know what you are trying to say then.

The brain has over millions of years been tuned to speak languages with certain structures. Deviating from these structures is more taxing for the brain. True statement. But how on earth does that imply the brain isn't 'thinking' for the structures it is used to ? Do you say you did not think for question 1 just because question 2 was more difficult ?

As I said it's not relevant but if you wanted to know you could put in the bare minimum of effort into doing your own research. From Smith and Tsimpli's "The Mind of a Savant": "On the [Gapadol Reading Comprehension Test] Christopher scored at the maximum level, indicating a reading comprehension of 16 years and 10 months". They describe the results of a bunch of other language tests, where he scores average to above average, including his translations of passages from a dozen different languages.

> But how on earth does that imply the brain isn't 'thinking' for the structures it is used to ? Do you say you did not think for question 1 just because question 2 was more difficult ?

The point isn't to define the word "thinking" it is to show that the language capacity is a distinct faculty from other cognitive capacities.

Reply on news.ycombinator.com