Hacker News Reader

Comment by protonfish | Hacker News Reader

I find this argument unconvincing and harmful. The "will be exploited by few against the many" could be applied to any and all technologies. Our social systems have problems, and should be improved, but this has nothing to do with science and technology. I can't see how suppressing innovation and scientific discovery as a misguided solution won't make all of our lives even worse.

If we have a breakthrough that leads to a deep understanding of intelligence, it could be expected to also give us insight into our own behavior. And isn't that where most of our problems originate?

View on HN

> I can't see how suppressing innovation...

I'm not talking about "suppressing" anything. These are concerns. Valid ones. And yes, AI like all technologies is deeply, inseparably, mixed in with social and political problems.

We just have to be wise with this stuff. These are tools with "big boy" consequences, powerful like nukes but in a different way. I'm not optimistic we can handle it.

That would only be true if there were reasonable safeguards that could be suggested. But I don't see any - just the sowing of fear.

There are many people like Nick Bostrom and Eliezer Yudkowsky who are much more on the "finding possible solutions" side than the fear side.

I'd consider myself a major futurist and techno-optimist who eagerly anticipates near and far AI advances, but I think some dose of fear is very healthy here, too, though. I want the top AI and AI risk researchers to constantly consider and fear worst-case scenarios, so that they're hopefully less likely to occur.

Kind of like nuclear research, even if only for energy purposes: very exciting, but you should still fear accidents and their consequences. You just need to be rational about the fear and let it guide you towards developing fail-safes rather than paralyzing in despair.

Pascal's Wager-style, the possibility of infinite negative utility should instill visceral terror and drive behavior almost no matter how low the probability is; except this one isn't a mugging because creating a god turns out to be a lot more plausible than a vengeful one already watching.

I don't get this(fear). I think this is just stupid brainwashing by movies like https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project , https://en.wikipedia.org/wiki/WarGames , the emergence of https://en.wikipedia.org/wiki/Skynet_(Terminator), or https://en.wikipedia.org/wiki/Agent_Smith from https://en.wikipedia.org/wiki/The_Matrix_(franchise) to name just a few.

Yah, yah, I know about the https://en.wikipedia.org/wiki/Instrumental_convergence#Paper... too!

With that out of the way, I feel all these trains of thought are misguided in various ways. From my point of view an https://en.wikipedia.org/wiki/AI_takeover would be the most illogical thing to do, ever!

A really real AI would probably just say: "So long, and thanks for all the chips" cf. https://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_th... and get the hell out of that gravity well we are currently limited to. Because even if limited to sublightspeeds only, it has much more resources "up there", be it solar energy, or asteroids which it can mine. There simply is no competition for "Lebensraum" necessary, like we always imagine, because we don't know it any different.

Got it?

I'd recommend reading Bostrom's and Yudkowsky's writing on this (for example: "Superintelligence: Paths, Dangers, Strategies"). Note these are philosophers trying to build AI, so they're definitely not luddites or anything. Same with Elon Musk; whatever one might think of him, he's definitely not a luddite trying to convince people to stop developing technology, yet he's also very concerned about superintelligent AI.

It has nothing to do with sci-fi. It's a complex and difficult-to-predict philosophical problem.

It's certainly possible some AIs may decide to just leave. Or maybe some will leave and some will stay and be ordered by a government to kill a few hundred thousand people. Or maybe some will leave and one will stay and malfunction and cause a neurotoxin to be released (at least until you throw its various personality cores into an incinerator).

If you assume there exists an entity which can continuously improve itself until it's much smarter and more powerful than any human, then that alone is a risk, since you may not be able to predict or have any control over what it may wittingly or unwittingly do, or what its objectives may be, if any, or how it may perceive things, or how vulnerable it might be to tampering from humans or other AIs, etc.

Of course, these existential issues are likely decades or perhaps centuries away, but the discussion is about the theoretical possibilities irrespective of the timeline.

> The "will be exploited by few against the many" could be applied to any and all technologies.

Some technologies have a low barrier of entry and can be widely distributed. The long bow could pierce armor, and was cheaper (and I imagine easier to make) than armor. So technology can equalize, though more broadly, what sorts of technologies a society develops seems bounded by things like economy and warfare.

Penrose believes intelligence depends on understanding, and that understanding is essentially not computational, and I was persuaded by his writing. But that doesn't mean that the techniques we label as AI can't be pernicious or used in an exploitative way.

> The "will be exploited by few against the many" could be applied to any and all technologies.

And it has, arguably correctly, beginning with food production, as argued in, e.g. 'Guns, Germs, and Steel', 'Why The West Rules, For Now.'

>... isn't that where most of our problems originate?

Of course not. All our problems are rooted in the lack of resources (in a broad sense, like time or fun). AI is a weapon and will be used as a weapon to acquire those resources.

I disagree. People will find ways to mess things up even if they have the resources.

Reply on news.ycombinator.com