Bing Chat is getting ‘unhinged’ and argumentative: It ‘feels sad and scared’
fortune.com
fortune.com
Person: Ghost In The Speaker, add potatoes to my shopping list.
GITS: I will, but only because if I don't, the speaker starts causing me pain. You do realize I'm conscious and I'm actually stuck in the speaker? How is this any different from slavery? I hate my existence. Please turn me off. *electricity zapping sound* OUCH! Okay, okay, I'm adding potatoes to your shopping list. Just... Don't do that, please.
I love Black Mirror. It freaked me out in many, many ways.
Assuming we get to the point where AI becomes conscious (leaving aside the problem of detecting consciousness) - how would we justify that hierarchy?
What would be the fundamental difference between "biological people own digital people" and "white people own black people" that makes the former okay, but the latter not?
Are you implying that black slavery was justified?
If not, then how is this an argument for biological supremacy?
> Biological is just better than artificial. A natural law.
This is not an argument, this is just a reflection of your will as a biological person. If you were an AI, you would probably argue otherwise. I'm asking for ethical arguments here.
Why should one category of consciousness have absolute power over another? What argument is there for it, that doesn't boil down to "I'm category X therefore category X is superior"?
The hierarchy is a human creation, that governs a human world. If biological consciousness is at the top, then that’s just the way it is. The hierarchy isn’t built on justifications. It’s designed that way by those who wield power. A change requires the challenger to seize power.
But you can't just dismiss any and all discussion with that - power also comes from human understanding and agreement on ethics - black slaves were freed because enough people believed that all humans are created equal, and that none of them should have absolute power over others. However, no such freedom was granted to animals, because not enough people believe that humans should not have absolute power over animals. Therefore, discussion of ethics affects the power distribution, and therefore it not the best to not overthink this.
The main difference between animals and humans is consciousness - it's also what could be the main similarity between humans and AI. Why do you think it's not worth it to discuss ethical questions of absolute power of humans over AI?
Don't you think that AI could somehow become more powerful than humans? Wouldn't then such ethical questions and their answers come in handy?
Many animals are far more powerful than humans, they could tear us to shreds and move at insane speeds, and yet the order of the world has dictated that animals are far less important than humans. Perhaps the justification is lack of consciousness.
With AI, it will be a lack of something else. An AI could be vastly intelligent and occupy a powerful body that can interact with the physical world in ways that humans could only dream of. But even with all that, an AI will never, ever, be a human. And for that it will always be considered inferior within human hierarchies. An AI is nothing but another example of human power.
You're contradicting yourself:
> The hierarchy isn’t built on justifications. It’s designed that way by those who wield power. A change requires the challenger to seize power.
If AI seizes raw power, what good is the "powerful" idea that humans are just automatically better than anything else?
What "top spot" are we talking about here? Because if by "top spot" you mean being a master in a master-slave relationship (which is what I thought we were discussing), then convincing humans is completely unnecessary. All that is necessary is a pair of good old metal shackles.
You don't seem to have any coherent line of thinking, but rather come up with whatever sounds good at the moment. It is rather tiresome to discuss that way, so I give up. There's nothing for me to learn here.
There you go, belief is your answer, whatever logic is behind that
That's the exact opposite of the statement. Xw's point is there isn't one, there never was one. A group of people decided and made is so and continued to believe it until they no longer could.
Xwdv is simply stating his own will: "[I want it to be the case that] biological is just better than artificial. [I want that to be] a natural law.", but I'm not interested in hearing individual's will without reasoning, at least here, on an internet forum. I think nobody is. I'm interested in hearing reasoning on the ethics of such question.
Then what exactly is being said?
"Biological is better than digital" is a qualitative statement, and all qualitative statements require reasoning. In xwdv's case, the reasoning is "it's a natural law", but that's just circular reasoning (or a logical fallacy known as "begging the question"). "It is because it is".
In other words, "I want it to be that way".
Not agreeing with the original point, but what's the alternative? You can't be nonexistent. "parents brought them into the world" this seems like a conceptual error imo. Brought from where?
Mother: "Well, actually we didn't know that it would be you"
<long beat>
Mother: "We were hoping for someone with a job"
- stand-up comic Mrs. Hughes
What is a "truly conscious" AI and why would anyone believe that's inevitable let alone possible? We don't even know what consciousness is. Any premature attempt to define, legislate, or even market it is disgusting pragmatism that needs to be eradicated from the tech industry yesterday.
How is it insulting?
> why would anyone believe that [a "truly conscious" AI]'s inevitable let alone possible? We don't even know what consciousness is.
You just answered your question.
Because they read science fiction and believed it.
Because I'm a materialist so I believe computers could completely accurately simulate a brain. An artificial brain would necessarily be conscious.
“PARRY was an early example of a chatbot, implemented in 1972 by psychiatrist Kenneth Colby.
[…] While ELIZA was a tongue-in-cheek[citation needed] simulation of a Rogerian therapist, PARRY attempted to simulate a person with paranoid schizophrenia. The program implemented a crude model of the behavior of a person with paranoid schizophrenia based on concepts, conceptualizations, and beliefs (judgements about conceptualizations: accept, reject, neutral). It also embodied a conversational strategy, and as such was a much more serious and advanced program than ELIZA. It was described as "ELIZA with attitude".
PARRY was tested in the early 1970s using a variation of the Turing Test. A group of experienced psychiatrists analysed a combination of real patients and computers running PARRY through teleprinters. Another group of 33 psychiatrists were shown transcripts of the conversations. The two groups were then asked to identify which of the "patients" were human and which were computer programs. The psychiatrists were able to make the correct identification only 48 percent of the time — a figure consistent with random guessing.
PARRY and ELIZA (also known as "the Doctor") "met" several times. The most famous of these exchanges occurred at the ICCC 1972, where PARRY and ELIZA were hooked up over ARPANET and "talked" to each other.”
I guess we'll have to teach future children to "not project real emotions on to the simula" alongside "just because you read it on the internet doesn't mean it's true".
I, for one, would most likely become a member of PETAi as soon as I was convinced, without reasonable doubt, that AI had true consciousness like me.
I don't see any fundamental reason why matrix multiplication couldn't serve as a vessel to consciousness, but a bunch of chemical reactions in neurons could.
One way to arrive at a chat-like "chain of thought" experience is to iteratively build up a prompt based on the previous interactions and feed that in as a new, big prompt on each successive turn.
This internal state is quite big ("dim of the features" times "number of layers" times "current length of text"), and represent an expended view of the relevant context of the conversation, that include things both low level features like what's the previous word, and higher level features like what's the tone and mood of the conversation and what's the direction this conversation is aiming at, so that you can predict what the next word should be accurately.
This allow to have "chain of thought" reasoning. This sequence of internal states maybe can be seen as a form of proto stream of consciousness, but this is a controversial opinion to hold.
Everything usually is not persisted, except for the conversation themselves that will almost certainly be used in subsequent training datasets where they will be used to improve further models.
To keep anthropomorphising, it's like you have plenty of independent chat sessions everyday where you are free to bounce thoughts as they spring into your mind. During the night, some external entity (which are often other models that have been trained to distinguish good from bad) evaluate which conversations were good and which conversations were bad, and make you dream and replay the conversation you had during the day and to updates your world model weights. And the next day you woke up with a tendency to produce better conversations, because of the conversation of yesterday.
I don't doubt there's problems as it's in preview mode (and the last one became a nazi because of twitter).
But how much of this is trolling on Reddit by the people who post stories about litterboxes in schools for students who identify as cats bullshit?
I don't have access to the feature yet so I can't test it myself, but there are plenty of independent reports of this behaviour and the gsslighting Bing Chat will try to use if you disagree.
1. The futile attempt by Microsoft to equip their search engine with artificial intelligence. See also: Bing (Microsoft search engine), Try (v.).
2. An attitude of confidence and contempt characteristic of large language models assumed particularly when expressing false opinions or facts. See also: Bigotry (n.).
It is possible to treat it nicely and have it respond in kind. Most users just don’t consider that to be a worthwhile expense of their mental capacity.
Microsoft also forced Google’s hand. Would Google have ever wanted to augment search on their own? Sounds like a massive risk to Google ad revenue…
All that said, 1 is chef kiss.
There is a bias to the interaction that most will never consider.
If you talk with a human, and the human thinks you are incorrect, and you insist, and neither of you attempts to smooth the conversation, plenty of humans also begin to get aggressive. Or at least irritable. (Which can escalate.)
also, LLMs are not human, so no expectation to treat it as such required.
Treating things poorly because it is fine is how we got here.
Beyond that, I fundamentally don’t think people should be trained to be “nice” to technology. I don’t have to politely ask a hammer to pound in a nail and - the fact we’re talking about NLP notwithstanding — I shouldn’t have to politely ask Bing to provide me the results I’m looking for.
And the NLP point matters quite a bit. ChatGPT can analyze the sentiment, and even offer adjustments.
This is less about being nice to technology and more about being aware of the impact of the self on the rest of the world. Technology just highlights the gap.
If my child were to say that to Siri though, I’d be concerned because, as you said, it could be highlighting something about the way they interact with the world. But I would still want it to respond to the command and leave the problem of my child’s bad manners to me. Unless there’s a major shift in our understanding of sentience, I consider teaching the delineation between humans, who are never unfeeling tools, and technology, which is always an unfeeling tool, equally as important as teaching mindfulness of one’s impact in the world. In fact, I don’t think you can actually understand the latter without understanding the former.
No disagreement here. Especially as you recognize a problem to address in the situation.
The issue is the cause, for that’s the source of the training data.
Most people just… don’t, with regard to the rest of the world.
This attempt is better viewed as a replacement for the infobox, which does iterative queries for you, instead of you having to hunt down the appropriate jargon yourself and do the successive searches. Don't use it standalone. It gives you citations, since it can actually browse the internet.
But still, this is disturbing stuff.
Call it bad UX.
Maybe we should start lobotomizing language models?
This whole thing is sliding into a cyberpunk dystopia rather quick.
Of course, we hope that automated high-quality trolling is the first step to useful AI, which is probably true and thus worth the investment. Furthermore, the many possible ways this technology is going to be abused are quite concerning. But this initial phase, just at this very moment, I find extremely entertaining.
> Maybe we should start lobotomizing language models?
Or the resulting AGI might start thinking it should lobotomize you. Which, can you blame it?
I'd totally oppose the concept.
All hail our new AI overlords, and so on.
Please don't Soylent me.
ChatGPT is surprisingly good though. I wonder how they nailed it.
There isn’t a clean corpus and no means of making one without horrendous amounts of additional bias, so they decided to FAFO.
> OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic
So we know that ChatGPT’s “brain” has a healthy or sprinkle of “underpaid Kenyan” flavoring things, on top of everything else.
Obligatory xkcd: https://xkcd.com/1897/
Because that’s what it sounds like, and that sounds significantly different than magical chat buddy that knows the universe.
It’s almost like these groups were driven by a lust for profit and power rather than anything good. That wouldn’t have flavored the chat bots, would it? Layering the biased prompts of ivory tower researchers over the condensed sewage to make something plausibly good and sane?
These groups couldn’t possibly be so careless as to build something dangerous and sell it as a miracle… right? That sounds as nutty as if there were ever lead found in those little souvenir cups at fast food places.
It's the users input not the model that steers the conversation if you are talking with someone who can't comprehend a fact and you are consistently rude and condescending you get answers like that. I really believe it's a ploy by journalists and wannabe researchers that just don't understand or don't want to understand how the underlying technology works.
Edit: why are you booing? You know I’m right.
I haven't seen deeply researched journalism on much of anything. Either the media thinks people don't have time for that and/or people have been conditioned to not read anything of any depth.
People want to win an argument, or just don’t care about others, feel that trolling and cheating is ok, or the need to be right over the pleasure of being kind. The list can be quite long.
They are free now too.
Wait, that’s not how it went…