Extinction by AI: A terrifying interview with ChatGPT | Hacker News Reader

Extinction by AI: A terrifying interview with ChatGPT

fernandoamaral.org

4 pointsfmfamaral9 comments

Person: writes "KILL ALL HOOOMANS" on a sheet of paper

Person: puts the paper on the glass of a photocopier

Person: presses "Copy"

Person: ZOMG THIS PHOTOCOPIER IS SENTIENT AND HOSTILE!!!!!!

"A 50% chance of survival It took a while for ChatGPT to put a number on the probability of an AI-driven catastrophic event. After a heated discussion, we agreed that the likelihood of annihilation is the same as winning the flip of a coin." This is way too scary!

GPT-4 is out and I’ve talked to it about the impact of general AI on the future of humanity. We explored the existential risk hanging over humankind and ChatGPT (using the persona Dr. Adrian Valerius) explained in great detail how everything can go terribly wrong.

I am 100% sure people will gaslight themselves with GPT and will do really serious stuff because of it. You didn't "talk" to "it" you instructed an LLM with intention and it gave you the answers you wanted.

It's not meant to be taken literally, it's meant to spark discussion. I gave it different "instructions" that it could have picked up on, but it didn't. But the interesting part is if the doomsday scenario sounds impossible or not given where we are today. I don't think it sounds impossible at all.

"didn't pick up on it"? You are on a bad path if you start humanizing an LLM. Just read the paper on how the tokenizer works so you at least understand how you are gaslighting yourself. The discussions you want to have been had and have been answered by people that actually thought about it in the 50-70s by Lem, Bradburry, Clarke, Asimov, Heinlein, Dick...

How am I humanising it by saying "it" didn't pick something and picked something else?

"It" is a LLM, not a person or anything like a person. "It" can still have negative effects on society.

It could not have "picked up on" that's humanization because it literally couldn't have answered your question differently. There is no choice.

Reply on news.ycombinator.com