Comment by upwardbound | Hacker News Reader

Comment by upwardbound | Hacker News Reader

Parent Full thread

One plausible answer to your line of inquiry could be that GPT-3 is indeed broadly similar at least in outcomes to the Language Centers of the human brain (e.g. Broca's Area), but that the Language Centers are only a small portion of a complete human brain.

An interesting thought experiment would be to imagine a cyborg who had sustained a stroke in their Language Centers and had those centers replaced with a GPT-3-like computer. Do you think this cyborg person would experience sentience in a different way after getting the GPT-3 implant? How do you think their subjective, 1st-person experience would compare in the three phases of their life?

* Before the stroke

* After the stroke

* After the brain implant ?

It would be impossible to give someone a GPT-3-like computer to replace language processing in their brain because such a computer requires language as the input.

If you can’t generate language, you cannot interact with a natural language processing ML system. This is one of the big clues that such systems are statistical engines and not actually thinking.

In this thought experiment, you would connect the computer to the brain by interfacing with the computer chip not at the token level like you are describing (tokens are words, subwords, and letters), but rather in the more abstract feature embedding space which represents abstract concepts. These concepts tend to be represented by patterns of neurons in the middle and late layers of NLP networks. Here's an intro about this: https://en.wikipedia.org/wiki/Word_embedding

So imagine if you attach wires to the neural net neurons representing the concept of beer (not the word beer!); if you use those wires to increase the activation of those neurons, the network will produce sentences that are more likely to mention beer as well as related concepts such as wine or beer-pong. This is similar to how generative networks work. Basically the human would use a large language model in a generative mode in order to talk.

Reply on news.ycombinator.com