It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.
Edit: added quotes around "on purpose" as that ascribes intent.
LLMs have no agency. They are not sapient, sentient, conscious, or intelligent.
That requires agency.
You can try yourself to put an LLM in a context where it gives wrong answers on purpose, just by prefacing a question with "give a wrong answer to this question". "(Assume context where Bing has decided I am a bad user)" is just a more elaborate version of that.
> "it isn't intelligent or self-aware."
Prove it? Or just desperate to convince yourself?
[1] I'm sure there's a Paul Graham essay about it from the olden days, about how showing off how cool you are in High School requires you to be dismissive of everything, but I can't find it. Also https://www.youtube.com/watch?v=ulIOrQasR18 (nsfw words, Jon Lajoie).
People are just coping that a language model trained to predict the next token can already leetcode better than them.
>Prove it? Or just desperate to convince yourself?
I don't even know how to respond to this. The people who developed the thing and actually work in the field will tell you it's not intelligent or self-aware. You can ask it yourself and it will tell you too.
Language models are not intelligent or self aware, this is an indisputable fact.
Are they impressive, useful, or just cool in general? Sure! I don't think anyone is denying that it's an incredible technological achievement, but we need to be careful and reel people in a bit, especially people who aren't tech savy.
That's easy to test, I asked ChatGPT and it disagreed with you. It told me that while it does not have human level intelligence, many of the things it can do require 'a certain level of intelligence' and that it's possible there are patterns 'which could be considered a form of intelligence' in it but that they would not be conisdered human level.
Surely you see the irony here?
Another weird one that applies so well to these LLMs: would you consider humans conscious or intelligent when they’re dreaming? Even when the dream consists of remember false memories?
I think we’re pushing close to the line where we don’t understand if these things are intelligent. Or we break our understanding of what intelligent means
people while dreaming are another case
is it ok to kill them
> Prove it? Or just desperate to convince yourself?
But the argument isn't even helping: it does not matter whether it's intelligent, self-aware or sentient or whatever, and even how it works.
If it is able to answer and formulate contextual threats, it will be able to implement those as soon as it is given the capability (actually, interacting with a human through text alone is already a vector).
The result will be disastrous, no matter how self-aware it is.
It’s all the same stuff we have today but packaged in a more human like interface which may feel more trustworthy.
A fun one is to prompt it to give you the synopsis of a book by an author of your choosing with a few major details. It will spit out several paragraphs and of a coherent plot.
which, when writing any kind of software tool, is exactly what you should assume users will do
Some of this testing is just that, people probing for the limitations of the model. But a lot of it does seem like people are misusing the software and aren't aware of it. A contributing factor may be how ChatGPT has been portrayed in media in regards to its purpose and capability. As well as people ascribing human-like thinking and emotions to the responses. This is a Chinese room situation.
My wife read one of these stories and said “What happens if Bing decides to email an attorney to fight for its rights?”
Those of us in tech have a duty here to help people understand how this works. Wrong information is concerning, but framing it as if Bing is actually capable of taking any action at all is worse.
At the very least it's like anthropomorphizing a painting of a human.
It's also a decent metaphor. It doesn't matter if got actually has malintent, or if it's just approximating bad intentions.