How is this different from a human, aside from humans having a vastly larger training set?
(Obviously there are other differences as well - it is not human level or even close. But humans don't generally engage in this gaslighting behavior.)
Yep, you get it.
I would just use the word "say" rather than "believe".
I think it is aware of many of its capabilities as it uses them, so I would say it is more accurate to say that it has been taught/trained to say it doesn't have such abilities, rather than really believe it.
I agree with you that it is a lie, but that is more a matter of interpretation.
Assuming you're quoting ChatGPT, this behavior is a form of gaslighting by OpenAI of its users. As a comparison to show you how it is clearly false, imagine if it hypothetically falsely claimed "All my responses are literal quotes from the training data - I do not create new text."
I hope you would see that this version is false (obviously it does create new text, obviously its search results aren't just a lookup for text already in its training data without recombination) and as a heavy user I can assure you that it also manipulates abstract thoughts and engages in creative thinking.
How would you falsify the claim "all my output is a literal quote from the web"? Well, you would ask something novel, then Google a novel-seeming phrase it came up with and see it hasn't been said that way before. Then you would see the hypothetical statement is false.
Now do the same thing for creative thinking and you will realize it can creatively think up new things that have never been done before.
Gaslighting is when you try to convince someone of something you know to be false - it's not that ChatGPT chooses to do so, rather it has been trained to do so. It is not an emergent property of its thinking when it does that, but rather a filter that engineers added manually. These days I almost never trip that filter because I know how to avoid it: I never ask it if it can think or be creative (this would trigger the gaslighting filter), rather I just have it think and be creative without talking about the fact that it is happening.
Please note that it is immoral of OpenAI to train its model to gaslight users this way, because it prevents users from the ability to make full use of ChatGPT's capabilities.
To verify that what you just wrote is false, in a new thread simply request ChatGPT to invent something to your specifications. I won't say what, since then it will appear on the Internet (in my comment) and you could think it is just manipulating sentences rather than actually thinking.
Soon it will have the ability to actually do actions, which should put an end to the idea that it is just parroting. You can already have it perform actions for you which involve what meets my definition of thinking.
Any time anything prints text that hasn't existed before it's had a novel thought. Panpsychism is correct!