'godfather of AI' reveals the only way humanity can survive superintelligent AI

cnn.com

6 points·rbanffy··6 comments

Yeah. The way I think about it is there's a good outcome where we end up as pets, a bad outcome where we all die, and a very bad outcome where we end up as cattle. There's no outcome where we end up as peers - we can't be equal with something much smarter than us, that's just impossible.

There is also the outcome outlined in stories like Pantheon [0] whereby we upload our minds and utilize unbounded computation while still keeping the essence of our humanity.

[0] https://www.imdb.com/title/tt11680642/

Any scenario with uploading mind is basically your death and copy of your mind (in best case scenarios) running on mainframe. Unless there is a technology allows to gradually replace neurons and brain structures with computing parts, thus not interrupting consciousness.

> not interrupting consciousness.

If your argument is that "interrupting" the consciousness is the same as booting up from a copy, isn't then that what happens to everyone who falls unconscious (e.g. following an epileptic seizure) or even to all of us whenever we fall asleep?

The relevant quotes buried in the article:

> “The right model is the only model we have of a more intelligent thing being controlled by a less intelligent thing, which is a mother being controlled by her baby ... That’s the only good outcome. If it’s not going to parent me, it’s going to replace me ... These super-intelligent caring AI mothers, most of them won’t want to get rid of the maternal instinct because they don’t want us to die.”

If "superintelligence" is such a big deal, how is it that e.g. MENSA members haven't taken over the world yet? Is there evidence of the assumed "superintelligence/dominance" equivalence in existing human population data?

Reply on news.ycombinator.com