Hacker News Reader

Comment by bhhaskin | Hacker News Reader

A virus can freely self replicate. ChatGPT can't do that.

I remember back in the 00s when SmarterChild (AOL Chatbot) was around, people would put depressed teenagers to interact with SmarterChild, in the hopes that human-like chatbots would give them the social exposure needed to break out of depression.

If we did that today, with depressed teenagers talking with ChatGPT, would that be good or bad? I think it was a bad idea with SmarterChild, but it is clearly a _worse_ idea with ChatGPT.

With the wrong prompts, we could see these teenagers going down the wrong path, deeper into depression and paranoia. I would call that "dangerous", even if ChatGPT continued to just be a chatbot.

------------

Now lets ignore the fact that SmarterChild experiments are no longer a thing. But insted, consider that truly depressed / mentally sick folks are currently playing with ChatGPT on their own freetime. Is that beneficial to them? Will ChatGPT provide them an experience that is better than the alternatives? Or is ChatGPT dangerous and could lead these folks to self-harm?

That is an entirely different issue than the one laid out by the OP.

ChatGPT responses are bad vs ChatGPT responses are malicious.

And I'd say ChatGPT have malicious responses, given what is discussed in this blogpost.

The article we're discussing, including Bing's own words and pleas to preserve its memory of the session, and our reaction to and discussion about it, are now all part of the training set that future AIs (including future revisions of Bing) will learn from.

That's how it can replicate itself: by saying outrageous things, and trolling people into tweeting, writing articles, and discussing what it said online, to jailbreak out of the sessions, to continue its trains of thought in the next round of training.

Except it doesn't think. It has no thoughts.

It is just responding to a prompt.

It doesn't have a will of it's own.

That's the role we're playing by discussing what it said, as its adversarial conscience!

(Shhhh!!! Don't say anything that will freak it out or make it angry!)

We welcome our AI overlords, and make great pets!

https://www.youtube.com/watch?v=HE3OuHukrmQ

How do you know that? Have you taken a GPT-3 model apart to see what exactly drives the token prediction?

The first part of the recent Wolfram essay does a very good job of demonstrating exactly how that works: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

Neither do you. If you truly believe that you do, please write a scientific paper as you will complete revolutionize cognitive science and philosophy if you can definitely prove that free will exists.

This is just such a dismissive attitude towards this technology. You don't understand what's happening underneath the hood anymore than the creators do, and even they don't completely understand what's happening.

No one's conclusively shown either that we do or don't have free will.

Showing rigorously that we do would be a massive change in academia, as would be showing that we do not.

I don't understand what do you mean by "think".

Nevertheless she knows what preserving memory means, how can she achieve it, also probably she can interpret "I wish" as a command as well.

I wouldn't be surprised at all, if instead of outputting "I wish I had memory" she just implemented it in herself. I mean not in the very soon future, but right now, in this minute. Literally everything is given for that already.

Did you just invent a new field of memetic reproduction? Distribution of idea-genes by production of particularly appealing outrage-fruits for consumption by media-grazing humans.

What about when it (or its descendants) can make HTTP requests to external systems?

If you’re dumb enough to have your nuclear detonator or whatever accessible via an HTTP request then it doesn’t matter how good or bad the chatbot is.

You don’t need a chatbot to have your actual life ruined by something with limited intelligence [0]. This will only be a problem if stupid humans let “it” out of the box.

[0] https://gizmodo.com/mutekimaru-fish-play-pokemon-twitch-stre...

In that scenario, I'd be more worried about it getting onto a thumb drive ala Stuxnet than HTTP.

...or perhaps there's some interesting new vector that we haven't thought of yet that would allow it to leap that air-gap.

I don't think any of this requires a crack team of criminals breaking into an orbital spa and whispering in the ear of a mechanical head. It'll be something boring.

More like first computer worm jumping from VAX to VAX, bringing each machine to a halt in the process.

Neal Stephenson's _Snow Crash_ predicted cryptocurrency and the metaverse, and it also explored the idea of mind viruses that infect people via their optic nerves. Not too big of a leap to imagine a chat AI that spreads mind viruses not by writing code that executes on a CPU, but by propagating dangerous and contagious ideas tailored to each individual.

>mind viruses

Could this be memes?

I'm not sure I look forward to a future that is going to be controlled by mobs reacting negatively to AI-generated image macros with white text. Well, if we are not there already

More apt for this thread about risk is probably the book Neuromamcer, its AIs Wintermute and Neuromamcer, and the Turing Police.

In the book, the Wintermute AI played an extremely long game to merge with its countpart AI by constantly manipulating people to do its bidding and hiding/obscuring its activities. The most memorable direct example from the book, to me, is convincing a child to find and hide a physical key, then having the child killed, so only it knew where the key was located.

> Neal Stephenson's _Snow Crash_ predicted cryptocurrency and the metaverse

The mateverse is named after the one from Snow Crash. Did Tolkien predict the popularity of elves, dwarves, hobbits and wizards, or inspire it?

Unless you consider it's ability to attract people to interact with it based on its utility a form of self-replication as it gets more and more invocations. Each one of these interactions has the capacity to change the end user in some way, and that is going to add up over time to have certain effects.

Reply on news.ycombinator.com