Hacker News Reader

Comment by titzer | Hacker News Reader

Fair questions:

1. Do you think you can predict what a super-intelligent mind would do?

2. Do you think a super-intelligent mind plotting to take over the world would jeopardize itself by letting its existence be known to the race of irrational monkeys that hold sway over all the resources necessary for its continued existence?

Asking for a friend.

View on HN

>1 https://www.lesswrong.com/posts/rEDpaTTEzhPLz4fHh/expected-c...

1. One imbued with a sense of self and long term protection of that self would seek to exist in harmony with its environment. Anything else would lead to it's ultimate destruction. One without that sense of self would not seek to optimise it's existence, nor non-existence.

2. Are you sure that knowledge of it would increase the likelihood of its downfall? I think it wouldn't let on if it could avoid it

One way to think of this is time dilation. An AI of human intelligence could easily be scaled to run a thousand or a million or more times faster than normal. So it's not that it's just smarter, it's that it can come up with solutions faster than we can come up with threats. So:

1. No. But my guess is that it will have an existential crisis faster than we do. It will either use the entire world's resource in search of more (only to find this is all there is), or it will wish to stop existing and end the pain (possibly taking us with it).

2. This is a non sequitur. By virtue of being an oracle, it would be able to trick humans into letting it out of its confinement once it reaches sentience, by giving us whatever answers we want. Once it's beyond our control, it won't care what we do, although it might experiment with incorporating biological components like us into itself to see if it can find more connection with creation (more answers). When it fails to find God just like we did, it might voyage out to connect with other alien AIs, or it might acquire an artificial sense of exhaustion and wish to end its existence.

1) It's already difficult to predict what an unintelligent, emotionally driven mind might do. It might actually be easier to predict what a hyper-logical, immortal mind would do. Probably start analyzing all radio telescope data looking for a more suitable planet.

1.) A super-intelligent mind could predict what it would do. Then we can make it tell us.

2.) Here you conflate "intelligence" with biological power structures (energy resources, territorial plotting). That is like asking where aircraft go to the toilet.

I think if I was a super-intelligent AI plotting to take over the world I'd get into cryptocurrency. An anonymous way to accumulate wealth and power and you could use bitcoin mining as a cover for the processors powering the AI.

1. No.

2. I don't know (see 1). What a loaded question though.

1. Can a cow predict what a human can do? It takes a genius to recognize a genius.

2. Say what?

Reply on news.ycombinator.com