Comment by naasking | Hacker News Reader

Comment by naasking | Hacker News Reader

Parent Full thread

> What the hell, we are looking at essentially millions of matrix multiplications done on powerful GPUs. And we're told this is what the brain does. Which is supremely stupid, since as Bernardo Kastrup has emphasized many times, our brains are completely UNLIKE those systems.

That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a transformer.

These transformers are learning how to reproduce functions by inferring associations between inputs and outputs. The data whose function is being reproduced is the function in the human brain that produces human text. Therefore these transformers are literally learning how the human brain generates text, ie. this is what the brain does.

Furthermore, there have been neuroscience papers correlating spiking behaviour in neural networks and human neurons when processing the same information, lending even more credence to this hypothesis. It's not as far-fetched as you think.

> That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a transformer.

That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain.

Linux is written in C and that‘s Turing complete. And? Therefore Linux is like the brain?

> That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain.

It's not irrelevant, it was merely the premise for the following point, which was that transformers are trained to learn a function. Since they're Turing complete, they can thus hypothetically implement any function, including the function governing human speech, as I described.

If transformers were not Turing complete, then they arguably would not be computationally powerful enough to model the brain which we know is Turing complete.

What is irrelevant is trying to dismiss transformers as intelligent, or like the brain, simply because they're "merely matrix multiplications".

What is your basis for asserting that transformers (not even sure what's meant: one layer?) are Turing-complete?

It's a well known property:

https://arxiv.org/abs/1901.03429

> What is irrelevant is trying to dismiss transformers as intelligent, or like the brain,

Well there is a very very very big thing that makes it relevant, which would be the fact that we have no fucking clue on how our brain actually works.

Wild claims, as always, no sources, as always

Thanks for agreeing with my point, which is that all of the people claiming that these systems are NOT like the human brain have no clue what they're talking about, and that these systems very well could operate just like the brain.

Winner winner chicken dinner. Could sum the argument as: Transformers appear to be approximately emulating the brain despite us not fully understanding how either works "inside". Which is weird and eerie, but plausible.

One smart ape once said: extraordinary claims require extraordinary evidence

The burden of proof is on you, and so far you produced nothing of value in that regard

The burden of proof is always on whomever is making a claim. I didn't claim transformer systems were like the brain, I simply showed the mistakes behind the arguments claiming the opposite, by showing how transformer systems could work like the brain.

> The burden of proof is always on whomever is making a claim

Yes, you in that case, you said "ie. this is what the brain does."

We don't know "what the brain does", so you can't make this claim

Do you have like any sources to back up you magical claims ?

The more I look into the more I see a cult/religion more than a science

What claims, specifically, do you want elaborated?

Transformers are turing complete: https://arxiv.org/abs/1901.03429

Additionally, Transformers are RNNs: https://arxiv.org/abs/2006.16236

Transformers are turing complete RNNs, as are the Neocortex.

There are differences like spiking vs non-spiking neurons, universal vs non-universal activation, the effects of brain waves, bicameralism, etc. But these differences may or may not be important depending on the topic of discussion. Additionally, the differences increasingly disappear as transformers are increasingly augmented with functionally equivalent systems.

Reply on news.ycombinator.com