Hacker News Reader

> Humans can perform math. Computers can perform math. No one would claim that's evidence computers think like humans or vice versa.

But we would very sensibly claim that computers can think like humans when suitably programmed, and humans can compute like computers. And the claim here is that learning the relationships between words is an understanding of language, and natural language reflects certain kinds of human cognition, and mimicking that output from the same input is mimicking that cognition.

> I'm going to make this very clear: there is absolutely no evidence that supports this claim. Period.

Again, that's incorrect. In what other science could you produce a model that nearly 100% accurately reproduces what the system being modelled would generate, and people would insist on saying that that doesn't really model the operation of that system? Inconsistent standards of evidence IMO.

In any case, There have been a few studies demonstrating strong correlations in activation patterns between the human brain and neural networks. These are correlations, but correlations are evidence.

Furthermore, I think you're failing to understand the argument. Human languages were invented by humans. They are necessarily suited to the human mind, reflecting some fundamental structure and operation of the human brain.

It would be a fairly dramatic coincidence if other, random formal systems were well suited to reproducing natural language. In fact, the most obvious inference is that LLMs are likely inferring semantic models that encapsulate how humans categorize and think, which is why LLMs can translate text between human languages. This would not be possible if languages did not have a common underlying semantic structure.

>> Again, that's incorrect. In what other science could you produce a model that nearly 100% accurately reproduces what the system being modelled would generate, and people would insist on saying that that doesn't really model the operation of that system? Inconsistent standards of evidence IMO.

As far as I'm concerned the problem with claims of great accuracy (100%? Surely you jest?) is that they go above and beyond the evidence provided by modelling. For example, take Convolutional Neural Nets used to train classifiers for objects in images. It is easy to agree that such systems model the human labelling of sets of pixels in digital images, but then this observation is taken as evidence that the systems in question are doing something more than that; namely, that they are somehow modelling human perception.

You can't just take any evidence you have and use it to support any hypothesis you like, and then call that "science". That's not science, it's wishful thinking. If you want machine learning to be scientific you have to be rigorous. But I'm sure if you start talking about scientific rigour in ICML and NeurIPs people will just laugh at you. "Here", they'll say, "we got systems worth millions, what's rigour got to do with anything?".

And they'll be right. Scientific rigour has never made anyone any millions.

> As far as I'm concerned the problem with claims of great accuracy (100%? Surely you jest?)

The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically and grammatically correct natural language sentences. This was not a claim that LLMs produce make accurate factual statements with near 100% accuracy. You have to specifically instruct or lead an LLM into violating syntactic and grammatical correctness.

> but then this observation is taken as evidence that the systems in question are doing something more than that; namely, that they are somehow modelling human perception

I think in the case of language this is more likely than not, because language was an invention of the human mind. Therefore the structure of natural language reveals some fundamental properties of the human brain.

Whether this also applies to vision is much less clear, not least because vision models are trained on 2D images and humans are embedded in a 3D space, and the contents of images are not products of the human brain the way language is. There is just no direct parity of reasoning here.

Edit: in some sense, cameras were designed to work well with human perception otherwise they'd have no utility to humans, so there is something about human perception that's revealed, but what's revealed is considerably weaker and more indirect than the natural language case.

You wrote a lot in your other replies and I will follow-up when I have access to more than just a phone and so can provide a more detailed argument and citations.

>> The system ostensibly being modelled here is our ability or produce natural language, so the 100% accuracy referred to it's ability to produce syntactically and grammatically correct natural language sentences.

I see this as overreach also. We have no idea what are the limits of human language ability, and cannot even say how much of human language LLMs can represent. It gets harder and harder to evaluate LLMs the larger they get because we have no better models of language, but that still doesn't tell us how good LLMs are as models of language, it just tells us earlier models are worse. If we still want to be scientific about it, at best what we could say is that a LLM (or any language model) models its training corpus, with some accuracy. That it models our ability to produce natural language? How can we know that, if we don't know what are the limits of that ability? Indeed, a major problem in machine learning research right now is the lack of good tests of language models' abilities.

Sorry for writing too much in the other sub-thread. I'm looking forward to your comments.

I agree with most of your points and I think there’s a good argument that CS should use different terms that don’t overlap with biology, but I don’t agree with this:

> there is absolutely no evidence that supports this claim. Period

The evidence is that it understands and can respond with human language which arose from purely biological processes. So until we understand how this happens we can’t say for sure that these things are not statistical models of part of the human brain. Maybe these models are picking up on Chomsky’s universal grammar or maybe they are emulating brains. We just don’t know.

It might not be strong evidence, it might be indirect, but it is not a total and irrefutable absence of evidence.

> The evidence is that it understands and can respond with human language which arose from purely biological processes

If you don't understand how this isn't evidence, I honestly don't know what to do.

> So until we understand how this happens we can’t say for sure that these things are not statistical models of part of the human brain.

I never said that.

I said you can't affirmatively say they are, and further, that because we don't understand how the human brain works, the fact that we can make computers perform tasks that humans can is not evidence in favour of the idea that those computers are in some way modelling the way the human brain actually works. And that is the claim I've seen many people make. In fact that's the claim you just made.

> We just don’t know.

On that we agree.

>> The evidence is that it understands and can respond with human language which arose from purely biological processes.

Well, human mathematical thinking, including logical thinking, also arose from purely biological processes (self-evidently) and yet we have machines that can reproduce all that: digital computers. Perhaps we should consider computers already artificial intelligence?

I actually think that yes, totally, 100% we should. But that makes for a definition of artificial intelligence that will disappoint most people who hope for Star Trek like computer-friends.

There is some evidence, though still tentative.

https://www.nature.com/articles/s41562-022-01516-2

(Evidence of a predictive coding hierarchy in the human brain listening to speech)

i view this as the newtonian mechanics vs actual reality debate. the former may not be a very accurate model of the latter, but it is very useful and it would be wrong to say there is no similarity between them.