Comment by chriswarbo | Hacker News Reader

Comment by chriswarbo | Hacker News Reader

Parent Full thread

Whether or not you're joking, I think the real problem with using Terminator as an example is that it's overly optimistic. The story is roughly:

1) US military builds up arsenal of autonomous killing machines and nuclear missiles

2) US military connects all of these to the Internet

3) US military creates a powerful AI which takes control of this arsenal (whether it was put in charge or hacks in seems to vary across the movies)

4) AI "becomes self-aware"

5) AI tries to wipe out humanity

Almost all of the discussion around this focuses on step 4, either by asking if/when an AI will "become self aware", or by trying to explain why that's meaningless and/or unlikely.

Meanwhile I think the real dangers are steps 1 and 2, which seem to be proceeding without much public outcry.

Yes, there are rogue AGI scenarios which end badly for everyone; but there are also issues of hacking (state-sponsored or otherwise), and/or terrorism (homegrown or otherwise).

It may have made political sense to build up ever-larger nuclear arsenals during the cold war, but these days it seems like that's just increasing the risk of accident or misuse.

You bring up the problem so many of us have with discussing issues at the micro level. I see the discussion of AI follow similar lines as the GMO debate. We often don't ask ourselves how these technologies play a part in a more larger system that appears to reward the concentration of power and Technics. Instead of asking ourselves if these things are innately good I feel that we ought to be asking ourselves what problems are we attempting to solve and how these technologies can affect those changes and if they are the best solutions.

There's less reluctance on the Russian side to build combat robots.[1] Policy from China is unclear, but swarms of 1000 drones have been demonstrated.

A reasonable near-term prospect is a package of maybe 1000 armed drones, programmed to kill anybody carrying a gun. Turn this loose on an occupied town, and in a few minutes, the occupiers have been thinned out enough that opposing troops can enter.

[1] https://www.researchgate.net/publication/309732151_Russia%27...

The even scarier scenario is:

0) AI "becomes self-aware," hides

1) US military builds up arsenal of autonomous killing machines and nuclear missiles

2) US military connects all of these to the Internet

3) AI by default has control of these

4) AI wipes out humanity in massive, overwhelming strike

Again, I don't think that's scarier, since step 0 is a) pretty meaningless and b) completely unnecessary. Our technology is capable of so much destruction (intensional or inadvertent) that it doesn't make much difference whether a human pushes the button or the button pushes itself; least of all whether the self-pushing button is "aware" that it's pushing itself.

Yeah, our weapons are really destructive now, but AI like that won't have mercy. If any nation fires nukes and nuclear war starts there is no way in practice all of the human race will wiped out, in theory yeah and then it doesn'matter who fired them, but that is only in theory.

Reply on news.ycombinator.com