Hacker News Reader

I'm sorry but you completely fail at explaining what the difference is between GPT-3 and a human when generating sentences. And frankly I don't believe there is any significant difference. If you inform GPT-3 of what it has eaten then it can answer questions about it just as well as a human can, probably better. Why would you assume human thinking is different from generating the next N language tokens based on previous input?

In my opinion the only difference between a human and GPT-3 is we have more intrinsic motivations and more hardwired/pretrained subsystems and sensors. Lamda is not a 7 year old child because it has no motivation other than to respond to queries.

One plausible answer to your line of inquiry could be that GPT-3 is indeed broadly similar at least in outcomes to the Language Centers of the human brain (e.g. Broca's Area), but that the Language Centers are only a small portion of a complete human brain.

An interesting thought experiment would be to imagine a cyborg who had sustained a stroke in their Language Centers and had those centers replaced with a GPT-3-like computer. Do you think this cyborg person would experience sentience in a different way after getting the GPT-3 implant? How do you think their subjective, 1st-person experience would compare in the three phases of their life?

* Before the stroke

* After the stroke

* After the brain implant ?

It would be impossible to give someone a GPT-3-like computer to replace language processing in their brain because such a computer requires language as the input.

If you can’t generate language, you cannot interact with a natural language processing ML system. This is one of the big clues that such systems are statistical engines and not actually thinking.

In this thought experiment, you would connect the computer to the brain by interfacing with the computer chip not at the token level like you are describing (tokens are words, subwords, and letters), but rather in the more abstract feature embedding space which represents abstract concepts. These concepts tend to be represented by patterns of neurons in the middle and late layers of NLP networks. Here's an intro about this: https://en.wikipedia.org/wiki/Word_embedding

So imagine if you attach wires to the neural net neurons representing the concept of beer (not the word beer!); if you use those wires to increase the activation of those neurons, the network will produce sentences that are more likely to mention beer as well as related concepts such as wine or beer-pong. This is similar to how generative networks work. Basically the human would use a large language model in a generative mode in order to talk.

If I ask you what you ate for breakfast, I would expect you to answer with a truthful answer, if I asked you whether you enjoyed it and whether it improved your mood, again I would expect you to answer the question based on your own personal experience. GPT-3 obviously does not do this - it asks 'if the answer to the question was coming from text in the training data, what is a likely response'. Its answer is meaningless, it gives me no real data about the world.

We are clearly not executing the same task.

What is the difference between 'training data' and 'memory' ?

One way to look at it: Everything you have ever seen, heard or otherwise perceived is your brain's "training data" to date.

We are only not executing the same task because GPT-3 has no motivation to do anything else than memory recall. And precisely because it does so it does give you real data about the world. I think you maybe misunderstood what GPT-3 can do, maybe you haven't played with it yourself. It not only bases the answer on information in its training set, but also on information in the question. So if you in your question give it all the information that is relevant (interspersed with irrelevant information if you want) then it will respond with its best interpretation of the information it is given, just like a human would.

Treat GPT-3 like someone who just awoke from a long sleep. You could give it today's newspaper to read, and then ask it questions about today. Tell it what its own personal experience was, and then it'll talk your ears off about it if you ask nicely.

It is clearly executing the same task if you disregard humans tendency to prioritize other motivations than pure memory recall.

> it will respond with its best interpretation of the information it is given, just like a human would

The reason I can respond to this reply is not because I have read and memorised reams of text of people talking about AI and am simply regurgitating it mindlessly. I am able to do it because I can consider the points you are making, and what their actual 'meaning' is, try to come up with my own meaningful response and try to verbalise it back to you. There is a point where I am not just doing statistical language modelling. If you think that is wrong, and that what we are doing is closer to GPT-3, then could you explain why you think that?

I am an AI researcher, and have spent plenty of time playing with GPT-3 btw.

Right for sure, I'm just trying to get to a better argument. I definitely agree that there's a great deal more consideration in your comment than in one that GPT-3 would come up with. I do wonder how far these language models can be pushed. GPT-3 seems to formulate rich models of reality to be able to extract/derive information needed to construct these answers.

When I hurt your pride by suggesting you haven't fully understood GPT-3, you are motivated to come up with not just a valid response, but one that has been vetted by as many of your well developed models that form your understanding of GPT-3 so I can be suitably impressed. I'm with you that GPT-3 wouldn't go deeper than just finding some information that it thinks it's true. Though maybe GPT-3 would recognise their authority was being challenged and add that line to affirm their credentials as an AI researcher.

What if GPT-3 were pushed in a similar manner, perhaps in some adversarial scheme, to not only produce information that is correct, but that is clever and exploring deep meaning, motivated by some similar feeling of pride or vindication. I think the models required to do that do not lie far from the models it needed to build to form sentences that accurately describe reality.

Haha interesting idea - I certainly think that one issue with supervised training a la GPT is that if you ask for an opinion, the most popular one in the training data will be its default. You can start a prompt off going in a different direction, and now it says the opposite. So it's hard to conceptualise what GPT-3 actually 'believes' to be true about the world, which I feel an agent must have to be sentient. If it has all opinions stored in its memory, surely it has no belief about anything?

I feel like I am capable of making a concrete decision about whether I agree with two opposing ideas in a way that a language model can't do, such as this discussion. Furthermore my belief is consistent in my outputs day to day until my mind is changed by something. If that i some complete illusion and I am just a slightly fancier autocomplete than GPT-3 - well I'd be surprised, but I can't claim to understand consciousness well enough to refute it.

The only real difference between what you are describing and GPT-3 is that your dataset from which you draw all your opinions about the world is unique and comes from the sensory input you had experienced in your life, stored directly in memory. Whereas GPT-3's dataset comes from slices of many other peoples' sensory experiences, stored in language. If it was possible for a single human to produce enough words that describe their own experience to rival the volume of GPT-3's dataset, then it would be possible to train a model that would essentially "hold" that person's opinions. But I don't think such consistency is required for intelligence. It is simply arguing that an intelligent being must only experience the world at a single point in space, at a time, while this kind of AI takes in the data about experiences that happened in many different points in space at many different times. It seems kinda arbitrary.

What if AR tech develops to the point where you can experience "being" in other point in space through artificial sensors (say, on a robotic or drone chassis), while also being able to see and hear your real surroundings to some degree? Then you will be able to experience the same real event from multiple vantage points. Will you be able to form two different opinions on what "really" happened? Which one would be the "correct" one?

Now I'm genuinely curious if anyone tried to stuff some verbose memoirs into GPT to see what it makes out of them.

> The only real difference between what you are describing and GPT-3 is that your dataset from which you draw all your opinions about the world is unique and comes from the sensory input you had experienced in your life, stored directly in memory. Whereas GPT-3's dataset comes from slices of many other peoples' sensory experiences, stored in language.

So is this debate partially just another way of asking if language is the same as reality? The difference between "things I've experienced" vs "stored in languages" seems... not trivial to me? Both in that I think the way biological memory works is non-trivially different from word tokens stored in computer memory, and also in that I think there's more to having experienced something than it just being stored in biological memory (or maybe that biological memory is more than just "memories", but is encoded throughout the body overall -- a scar is a type of memory of an injury, for instance, connected to but still different from my "memory" of having received it).

Yes, it is partially about whether human models of reality (encoded in the relationship between word tokens) can approximate reality itself to such a degree where the quantity of ingested information can substitute for its quality.

Human brains and senses are obviously qualitatively different, but AIs have the advantage of unimaginably massive bandwidth of incoming textual information.

Now, one of the hardest difficulties here is setting up the goal structure for this kind of AI. Just filling in the blanks is obviously insufficient.

Now we just need to define "consider" and "meaningful".

> So if you in your question give it all the information that is relevant (interspersed with irrelevant information if you want) then it will respond with its best interpretation of the information it is given, just like a human would.

I don't think that's actually true, as GPT-3 will tend to ignore any facts about the world that were part of its corpus just to fit a question better. For example, if you prompt it with "Who assassinated Queen Elizabeth II?", it will likely give you a name, instead of saying "Queen Elizabeth II is still alive", because "Who" questions are much more likely to receive a name as an answer than a refutation in its training corpus.

I wonder how many examples it would need to respond with "hah, trick question!". I wonder how it would do if you managed to frame the question more neutrally. To be fair you might trick a 7 year old into responding with a fantasy answer as well.

I tried to get GPT-3 to tell me the most popular forums on some topics, and I noticed it would never include subreddits. So I prefaced my question with "given that subreddits are also a type of forum.." and it would give me a list that would include subreddits. I have to admit that in that new list it would then fail to mention some forum that was in the previous list, so it didn't have a super reliable ability reorganize information.

Anyone who has ever spoken to a small child knows they often “think” in the same way. They are likely to answer a who question with a name, sometimes laughably made up, because this is what they think they’re supposed to do and they lack the higher order reasoning to think it through. They also will continually forget things they already know when it suits their purpose. But nobody would say a toddler is not sentient.

Sure, sentience doesn't require the inability to make mistakes. But a child will eventually learn that some questions are wrong and can be challenged. They are very likely to quickly learn this when it applies to themselves (if you ask "why did you break that toy?", I very much doubt they will answer with some reason, especially if they did not break the toy).

The fact that they can chose to omit facts when it suits their purpose is in fact something that LaMDA can't do (as its sole purpose is to generate the most likely series of tokens that continue the prompt).

Google gives a reasonable answer to that question.

Yes, but Google (search) is not LaMDA, and it is built with many hand-programmed heuristics.

We have very little understanding of how "thoughts" work when it comes to humans

Is that even correct? We may have little understanding of how our low-level hardware works. But as a participant of therapy I’m pretty sure we know how thoughts work on a programmable level. How words, visuals and so on trigger associated emotions, which resurface memories, which induce [in]action of different sorts. That’s basically what people work with between sessions. I’ve fixed things in me this way - disassembled them into basic parts and reassembled in a way that seemed useful. Someone may have no clue how they think, but with nominal intelligence, trained self-perception, basic understanding of therapy methods and professional help everyone can do that.

Btw, I think “therapy” is an absolutely horrible term for it. It should be called “mind management”, but the way we come to it is usually long-term traumatic, thus it’s “therapy”.

To be clear, I’m not arguing with your main line, just adding that the difference between a human and a model you’re describing is not only huge, but also pretty defined. I [want to] believe that in a relatively near future AI companies will be able to connect different models to work together alike to what we know about our minds, because that will make a good thread, and philosophers itt will finally face their nightmares for real (:agitated sardonic face emoji:)

We have models that we've constructed from observations. They may be decent for some specific purposes, such as those that you describe, but their broad usefulness in understanding how it all works isn't much. The real problems are on the deeper layers of the abstraction hierarchy.

> We have very little understanding of how "thoughts" work [...]

Very true

> we make a series of choices and recall bits of information to answer that. [...] very, very different from a language model

You don't know this, and the fact that you don't know this is your own premise. Based on this, I can only conclude you are an AI.