Hacker News Reader

Comment by lubesGordi | Hacker News Reader

I heard him on Lex too, and it seemed to be just a given that AI is going to be deceptive and want to kill us all. I don't think there was a single example of how that could be accomplished given. I'm open to hearing thoughts on this, maybe I'm not creative enough to see the 'obvious' ways this could happen.

View on HN

This is also why I go into chess matches against 1400 elo players. I cannot conceive of the specific ways in which they will beat me (a 600 elo player), so I have good reason to suspect that I can win.

I'm willing to bet the future of our species on my consistent victory in these types of matches, in fact.

Again, a given that AI is adversarial. Edit: In addition, as an 1100 elo chess player, I can very easily tell you how a 1600 player is going to beat me. The analogy doesn't hold. I'm in good faith asking how AI could destroy humanity. It seems given the confidence people who are scared of AI have in this, that they have some concrete examples in mind.

No it’s a given that some people who attempt to wield AI will be adversarial.

In any case a similar argument can be made with merely instrumental goals causing harm: “I am an ant and I do not see how or why a human would cause me harm, therefore I am not in danger.”

People wielding AI and destroying humanity is very different from AI itself, being a weird alien intelligence, destroying humanity.

Honestly if you have no examples you can't really blame people for not being scared. I have no reason to think this ant-human relationship is analogous.

And seriously, I've made no claims that AI is benign so please stop characterizing my claims thusly. The question is simple, give me a single hypothetical example of how an AI will destroy humanity?

Sure, here’s a trivial example: It radicalizes or otherwise deceives an employee at a virus research lab into producing and releasing a horrific virus.

The guy at Google already demonstrated that AIs are able to convince people of fairly radical beliefs (and we have proof that even humans a thousand years ago were capable of creating belief systems that cause people to blow themselves up and kill thousands of innocent people).

P.S. I was not characterizing your opinion, I was speaking in the voice of an ant.

If AI is general, it will also have the capability of wielding itself, and probably better than humans.

Other caveman use fire to cook food. Fire scary and hurt. No understand fire. Fire cavemen bad.

Other caveman use nuke to wipe out city. Nuke scary and hurt. No understand nuke. Nuke caveman bad.

Other caveman use anthrax in subway station. Anthrax scary and hurt…

Is AI closer to fire or closer to nukes and engineered viruses? Has fire ever invented a new weapon system?

By the way: we have shitloads of regulations and safety systems around fire due to, you guessed it, the amount of harm it can do by accident.

IMHO the argument isn't that AI is definitely going to be deceptive and want to kill us all, but rather that if you're 90% sure that AI is going to be just fine, that 10% of existential risk is simply not acceptable, so you should assume that this level of certainty isn't enough and you should act as if AI may be deceptive and may kill us all and take very serious preventive measures even if you're quite certain that it won't be needed - because "quite certain" isn't enough, you want to be at "this is definitely established to not lead to Skynet" level.

And even with all that, probably it's best to still exercise an abundance of caution, because you might have made a mistake somewhere.

Reply on news.ycombinator.com