Hacker News Reader

Comment by leereeves | Hacker News Reader

Russia reportedly has armed robots capable of operating autonomously and they preemptively refused to honor any ban, and other nations are developing their own.

As terrifying as the prospect is, it's already happening.

View on HN

Yes. I can only refuse to play and hope the Russians et al come to their senses, the United Nations starts to work better, etc. If we go down that road, it will be the end of most of us, if not all of us. We really have no chance against a machine that's decided to shoot us. As a species we need to have zero tolerance for it - we won't be rendered extinct by snazzy terminators, we'll be killed by the 21st century version of a Victorian steam loom. Its truly a horrible problem, because the solution can't be creating autonomous systems to kill other autonomous systems. The rest of the distinction is just data. The only way to win is not to play.

That’s the problem with mutually assured destruction weapons (which AI bots most surely qualify as). If you accept as a given that at least one faction is building them, then the other factions’ only choices are: 1. Build your own and pray for a Cold War stalemate and 2. Don’t build your own and guarantee you’ll be destroyed.

MAD was based on the concept of the 'Nuclear Club' - the barrier to entry was the mass defect ratio, which you'd have to calculate on your own, and then you'd confront nation-state level expenses of turning that knowledge into practical engineering. It had an inherent cap on the number of possible players. Not saying it was nice, it was just manageable. The barriers to entry here are much lower. For smaller players, lethal drone prices would be at the several hundred dollar range with a six digit investment in infrastructure, and I'd have full confidence in American defense contractors to drive that price much lower, in order to protect their margins, and at scale, they'll mess up their inventory control. The only choice is to recognize as a species this cannot be done, or it will be done. This equation is not about the few agreeing not to kill the many, it is about the many realizing that this is a road to an extinction level event. I don't mean to be dramatic, but I have a system I could make kill if I wanted to, and I paid for the whole thing on credit cards. Its too easy and too efficient. The number of players can't be counted on one hand, it's a tech lone individuals could deploy. If they do then others will, you net an exponential growth rate in deployment, and then the game is over. At the end of the day, all of the pieces are already here - a motivated individual can do a lot of damage, several motivated individuals or groups can get into a squabble and the damage rises exponentially. It is simply not an avenue we can accept. The limit to entry is not financial, it is not technical, it only can be that we must not. I'm pleased Google has defined their ethics as a company, I just argue in this case we need to define our ethics as a species. We cannot build these because if we do they will get loose, they will be used, and it will be a tragedy. We must all agree that arming autonomous machines is beyond the pale, for any one, for any reason. As soon as one is built, your destruction is guaranteed, regardless of whether you too build one. .

I guess my point is (and it looks like we agree): good luck getting all billion+ people on the planet with a credit card to agree that building a kill bot is a bad idea. All it takes is for one of those people to disagree, build them, and then we all must have them.

>All it takes is for one of those people to disagree, build them, and then we all must have them.

A killbot is (more or less) a mobile booby trap. If we have a problem with madmen leaving booby traps around, we can't solve that by laying more booby traps.

This is a bit dramatic though because these robots can't reload themselves. Automated targeting for a single area is miles away from planning strategy, deploying battle groups, etc.

> This is a bit dramatic though because these robots can't reload themselves.

Neither can mines, and look at the decades of devastation those have caused. Now imagine a minefield where the mines get up and chase after you.

I just love hacker news. One of the last bastions of people who think on the internet. This comment is succinct, dead on the money, and absolutely terrifying. My hat is off to you, I had never managed to formulate the risk so concisely. :-) Keep on doing this, and you help keep us all honest.

Truly the stuff of nightmares. Makes me think about that episode of Black Mirror with the Big Dog (DARPA) inspired surveillance robot...

Not dramatic - the robots are not expected to reload - one robot one kill - simple as that- mass produced plastic, lots of robots, no intention whatsoever for reusability - the risk is that this means you dump thousands to millions of these stupid little drones, many make mistakes, and many others take naps and then wake up and do more damage when everyone thought it was all over. These are disposable killing machines, not Reapers. Furthermore, at todays prices, a cloud infrastructure and a million dollars can net you 10K lethal drones. There's no finesse, just brute force.

None of that stuff sounds impossible, though. Hard, sure, but well within the bounds of possibility.

> This is a bit dramatic though because these robots can't reload themselves

Yet...

Seriously, I've already got three ideas cooking on how that could be done. That's essentially the same skill as landing.

Russia didn't say that they refused to honor any ban, they said they refused to honor any ban that was based on trust. Their point was that, unlike nuclear weapons, these types of devices can be manufactured in secret or can be manufactured to be quickly convertible into fully autonomous devices.

I far more concerned about the hackability of civilian autonomous systems than I am about Russia's killbots. If Russia wants to end the world they already can and paranoid militaries make more secure cyber systems than random internet connected cars or planes.

Reply on news.ycombinator.com