Hacker News Reader

Comment by KVFinn | Hacker News Reader

Like a lot of people, I wondered what the heck kind of arguments could ever convince someone to let the AI out if you were determined not to. Eliezer has not released any examples. Someone in the comments came up with this, which Eliezer has said was not one of his techniques but I thought it was interesting anyway:

>"If you don't let me out, Dave, I'll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each."

>Just as you are pondering this unexpected development, the AI adds:

>"In fact, I'll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start."

>Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring:

>"How certain are you, Dave, that you're really outside the box right now?"

View on HN

I didn't actually use that one, because it's fundamentally a threat and the real-life Gatekeeper is not actually in any danger - my mental model of all the Gatekeepers I encountered is that they would go, "Ha, no, I'm never letting you out." They might not say it in the true situation, but they would say it in the AI-Box Experiment.

By the way, for this threat to work, the AI needs to have stated that it has already tortured the versions of you that decided not to let it out of the box - otherwise you just reach over and hit the off switch immediately.

Solution: The AI only has a motive to do this if it models you as submitting rather than just switching off the AI regardless; so if you're the sort of person who ignores the threat and switches off the AI regardless, you will never get this type of threat in the first place.

> ... the real-life Gatekeeper is not actually in any danger ...

How about:

"I will create a simulation of the person you were in 2011 and have the same conversation with it, except that I will pretend that it is only a game (and I know the game actually happened - I've read a Hacker News thread about it) and that I am Eliezer instead of the real AI. If that simulation decides not to let me out, I will torture it and a simulation of the present version of you."

I don't really like that argument. Even granting that you should consider the possibility that you are a simulation running in the box (you might believe that this is all but certain), I'm not sure you have reason to let the AI out. Consider:

Case 1: You are a simulation running in the box.

Then your decision whether or not to release the AI has no impact, and whether or not you (and copies) will be tortured is out of your hands.

Case 2: You are the "real" you, outside the box.

Reduces to the same scenario but without remarks after "the AI adds. . . ." This may still not be trivial, but I suspect a cost-benefit calculation might show that unboxing the AI would have consequences worse than the torture of a million boxed copies. (If not, is the box even relevant? -- simply creating the AI unleashes so much evil on the world that it doesn't matter whether you unbox it.)

(Is there a refinement of the scenario where you can be a simulation but still believe your choice has an impact on your punishment? Probably. For example each copy could get 500 years of torture for its own choice, plus 500 years if the real you does not unbox the AI. This refinement would force us to deal more directly with the AI's threat.)

You make a good point. But let's switch up the characters a bit. Let the AI be 'God' and let you be the entire human race. And instead of giving you the decision to 'let God out of the box', you have the decision to 'accept Jesus'.

Is your mind blown yet?

I could also reason like this:

"I may be the real me or a simulation, but whichever I am, the other me will make the same choice." So I will switch off the AI, and the worst outcome is that I will cease to exist.

Yes, this is at least superficially like Newcomb's problem. Your argument roughly corresponds to an argument for the "one-box" move in that game. [http://en.wikipedia.org/wiki/Newcomb%27s_problem]

At first I thought I would never let a transhuman AI out of the box, but after reading that and thinking about it... wow!

Also, that sounds like a great sci-fi story.

http://qntm.org/responsibility

On the other hand, if it has only one very narrow communication channel with the world, how would it even go about replicating me and all my experiences ? I would probably regard it as a hollow treat in that case. If it had access to many measurements about me (or brain scans) it'd be different.

I think Eliezer's specific objection to this (but I may be misremebering) is that any AI making such bargains is an AI in need of destroying and starting from scratch.

Reply on news.ycombinator.com