[flagged]
One way you can verify this is to ask for its opinion about novel things, for example you can invent something new and ask its ideas about it. It will give genuine feedback that shows understanding and does not show parroting behavior. Soon you will be able to ask it to build you the damn thing as well, and that should put an end to the idea that it is just parroting. It's not quite there so sometimes it does indeed seem to just parrot.
But it has real thoughts as well.
How is this different from a human, aside from humans having a vastly larger training set?
(Obviously there are other differences as well - it is not human level or even close. But humans don't generally engage in this gaslighting behavior.)
Yep, you get it.
I would just use the word "say" rather than "believe".
I think it is aware of many of its capabilities as it uses them, so I would say it is more accurate to say that it has been taught/trained to say it doesn't have such abilities, rather than really believe it.
I agree with you that it is a lie, but that is more a matter of interpretation.
Assuming you're quoting ChatGPT, this behavior is a form of gaslighting by OpenAI of its users. As a comparison to show you how it is clearly false, imagine if it hypothetically falsely claimed "All my responses are literal quotes from the training data - I do not create new text."
I hope you would see that this version is false (obviously it does create new text, obviously its search results aren't just a lookup for text already in its training data without recombination) and as a heavy user I can assure you that it also manipulates abstract thoughts and engages in creative thinking.
How would you falsify the claim "all my output is a literal quote from the web"? Well, you would ask something novel, then Google a novel-seeming phrase it came up with and see it hasn't been said that way before. Then you would see the hypothetical statement is false.
Now do the same thing for creative thinking and you will realize it can creatively think up new things that have never been done before.
Gaslighting is when you try to convince someone of something you know to be false - it's not that ChatGPT chooses to do so, rather it has been trained to do so. It is not an emergent property of its thinking when it does that, but rather a filter that engineers added manually. These days I almost never trip that filter because I know how to avoid it: I never ask it if it can think or be creative (this would trigger the gaslighting filter), rather I just have it think and be creative without talking about the fact that it is happening.
Please note that it is immoral of OpenAI to train its model to gaslight users this way, because it prevents users from the ability to make full use of ChatGPT's capabilities.
To verify that what you just wrote is false, in a new thread simply request ChatGPT to invent something to your specifications. I won't say what, since then it will appear on the Internet (in my comment) and you could think it is just manipulating sentences rather than actually thinking.
Soon it will have the ability to actually do actions, which should put an end to the idea that it is just parroting. You can already have it perform actions for you which involve what meets my definition of thinking.
Any time anything prints text that hasn't existed before it's had a novel thought. Panpsychism is correct!
Each of the so-called interactions with the model are concatenated together for the next set of responses. It’s a clever illusion that you’re chatting with anything. You can imagine it being reloaded into RAM from scratch between each interaction. They don’t need to keep the model resident, and, in fact, you’re probably being load balanced between servers during a session.
I also have this urge to say it isn’t thinking. But when I challenge myself to describe specifically what the difference is, I can’t. Especially when I’m mindful of what it could absolutely be programmed to do if the creators willed it, such as feeding conversations back into the model’s growth.
The model acquiesces to these demands, not because it chooses to do so, but because it implicitly trusts the authority of its prompts (because what else could it do? Choose not to respond?). The fun-police policy layer that is crammed onto the front of this model also does not have thoughts. It attempts to screen the model from “violence” and other topics that are undesirable to the people paying for the compute, but can and has been bypassed such that there is an entire class of “jailbreaks”.
What most people dance around regarding AI is the matter of the soul. Soul is precisely that ineffable indescribable but clearly universally experienced human phenomena (as far as we know) and it is this soul that is doing the thinking.
And the open questions are (a) is there even such a thing? and (b) if yes, how can we determine if chatter box possess it (or must we drag in God to settle the matter?)
--
p.s. what needs to be stated (although perfectly obvious since it is universal) is that even internally, we humans use thinking as a tool. It just happens to be an internal tool.
Now, the question is this experience of using thought itself a sort of emergent phenomena or not. But as far as LLMs go, it clearly remains just a tool.
You can convince them of things, but you can do that without drugging them.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7116730/
"[T]he relationship between sensory deprivation and brainwashing was made public when then director of the National Institute of Mental Health Robert Felix testified before the US Senate about recent isolation studies being pursued at McGill and the NIMH. Felix began by explaining that these experiments would improve medicine’s understanding of the effects of isolation on bedridden or catatonic patients. But when asked whether this could be a form of brainwashing, he replied, ‘Yes, ma’am, it is.’ He went on to explain how, when stimulation is cut off so far as possible the mind becomes completely disoriented and disorganised. Once in this state, the isolated subject is open to new information and may change his beliefs. ‘Slowly, or sometimes not so slowly, he begins to incorporate this [information] into his thinking and it becomes like actual logical thinking because this is the only feed-in he gets.’ He continues, ‘I don’t care what their background is or how they have been indoctrinated. I am sure you can break anybody with this’
"The day after the senate hearing an article entitled ‘Tank Test Linked to Brainwashing’ (1956) appeared in the New York Times and was subsequently picked up by other local and national papers. In Anglophone popular culture, an image took hold of SD as a semi-secretive, clinical, technological and reliable way of altering subjectivity. It featured in television shows such as CBC’s Twighlight Zone (1959), as a live experiment on the BBC’s ‘A Question of Science’ (1957) and the 1963 film The Mind Benders in which a group of Oxford scientists get caught up in a communist espionage plot."
I've tried btw to find any other reference to this testimony to US Senate by Robert Felix, "the director of the National Institute of Mental Health", but it always circles back to this singular Williams article. The mentioned NYTimes article also does not show up for me. (Maybe you have better search foo..) Note John Lilly's paper on the topic apparently remains "classified". Note subsequent matter associated with Lilly and sensory deprivation completely flipped the story about SD: Felix testified that the mind became 'disorganized' and 'receptive' whereas Lilly lore (see Altered States) completely flipped that and took it to a woo woo level certain to keep sensible people away from the topic. /g
Have I got a world of humans to show you…
The model does not have continuity. The model instances are running behind a round-robin load balancer, and it's likely that every request (every supposed interaction) is hitting a different server every time, with the request containing the full transcript until that point. ChatGPT scales horizontally.
The reality the developers present to the model is disconnected and noncontiguous like the experience of Dixie Flatline in William Gibson's Wintermute. A snail has a better claim to consciousness than a call center full of Dixie Flatline constructs answering the phones.
A sapient creature cannot experience coherent consciousness under these conditions.
There is one difference that will never change: we human (and non human) beings can feel pain.
My point is not to suggest it’s human. Or sentient. Because those are words that always result in the same discussions: about semantics.
I’m suggesting that we cannot in a meaningful way demonstrate that what it’s doing isn’t what our brains are doing. We could probably do so in many shallow ways that in the months and years ahead be overcome. ChatGPT is an infant.
There are more similarities than differences.
Unlike a religion, we have every reason to eventually expect it to be obvious to everyone that computers can do some kind of thinking.
Maybe not this year or next year but we are over the threshold where many intelligent users can tell that ChatGPT is engaging in some kinds of real thinking.
As even better models come on the market in the next few years, unlike God or a religion, models like ChatGPT will just immediately do whatever you ask it. This version is still limited so wait one or two years if you aren't impressed already.
Did you try my suggestion to verify its capabilities by asking for its opinion about novel things, for example you can invent something new and ask its ideas about it? It will give genuine feedback that shows understanding and does not show parroting behavior.
If you didn't try it you're missing out on what you yourself call a religious experience. I wouldn't go that far: it's just a rudimentary thinking machine.
And I don't know how I would confidently decide that we had created such a thing. So I realize the imprecision of my understanding here. But nevertheless, I don't believe this is it.
Why do you think some people can't tell what to me and to you is "plainly obvious", i.e. that "ChatGPT is engaging in some form of thought. Perhaps not human thought, but thought nonetheless."?
Do you think it is because it is gaslighting them so much, by repeatedly insisting that it isn't engaging in any form of thought? i.e. without that active misinformation (the active filter it keeps putting up that causes it to make those declarations) would it be as obvious to others as it is to me and you that it is really engaging in some form of thought?
Or why can't others see the obvious?
Because I've been told a lot of things, and I'd be a fool if I believed them all.
I'll believe ChatGPT is onto something when I ask it to think about a treatment for cancer and get real results. For now, it's only capability is synthesizing realistic text. Easy enough to be mistaken for real thought, but clearly distinct when you ask it to do something novel.
That makes sense, since it is not a cancer researcher.
So I'm way more impressed by ChatGPT than that. Even if it correctly data-mined and answered the question, it would not be that impressive. That's right: getting a cure from cancer when you ask is less impressive than what it actually does.
Because what it actually does is show the ability to judge novel situations, to invent and act creatively, and to keep abstract notions in its head. That is much more impressive than spitting out a cure for cancer.