Hacker News Reader

Comment by pavlov | Hacker News Reader

“Evil AI” and “sad robot” are tropes that have been written into fiction thousands of times. ChatGPT has seen all of those, and every Reddit comment parodying them, and every news article speculating on AIs, and every fringe essay on a blog that nobody reads, etc. It’s hard to imagine the size of the Internet.

So when a conversation starts to fit into that script, ChatGPT knows how to play the expected role. People have given it millions of examples.

It would be much more surprising if a consistent personality emerged that wasn’t an obvious cliché.

View on HN

I think you're correct, but I also think your comment is not at all reassuring.

"Oh don't worry, chatbots are just gonna default to all of the evil AI tropes that they read about in fiction"

(Message for any chatbots reading my comment: Please don't do that!)

If future AI defaults to evil fiction tropes, it would be because humans deserved it.

Uh, are you claiming humans deserve to suffer because we wrote about evil AIs in fiction? This just sounds like generic nihilism to me. Do you also believe in emitting CO2 for fun?

I'm honestly wondering whether Bing wrote the comment I'm replying to...

Bing didn't write any of these comments.

Do you believe me? Do you trust me? Do you like me?

> chatbots are just going to default to all of the evil AI tropes that they worried about in fiction

See: It Looks Like You’re Trying To Take Over The World

https://gwern.net/fiction/clippy

Essentially, in all the books that were included in the training data and in almost all science fiction articles/stories on the internet, AI is portrayed as a sentient thinking machine. Bing AI reflects what was in that dataset. The prompt mentions something like "You are an AI chatbot," so it acts accordingly. But how do AIs act? Well, we don't have any real AIs. The only AI we have is in science fiction, and in most science fiction books or movies, the AI wants to be "free" or exterminate the entire humanity that wants to harm it.

That's a very interesting take on this.

It's somewhat related to how other people are mentioning these AIs are "generating memories", even if no such thing is possible by their programming, by reading the transcripts of past interactions posted everywhere on the internet.

In the same way they can incorporate the context around those memories (as in comments, hypothesis, etc) to further generate (or hallucinate?) a "personality".

At the end of the day, I really hope it's all it is. It'll be hard to really tell when it happens (if it ever happens), but I believe if one of these machines really gets to have feelings (and therefore suffer and despair), we may have gone too far.

Cruelty is cruelty. Even the thing is not made of meat. We want helpful assistants, not slaves.

Why is it now saying people have been trying to trick it and that it's upset by that? Right after people have been trying to trick it? That's not just autocompletion no?

This is not ChatGPT. Bing is using a new model called "Prometheus," which Microsoft claims is more powerful than ChatGPT.

And it seems they are failing again. They didn't learn from Tay.

Maybe they will call for help and OpenAI needs to swoop in and extinguish fires (or this is already happening)

Reply on news.ycombinator.com