Hacker News Reader

Comment by danso | Hacker News Reader

> Russell said it took him five minutes of Internet searching to figure out how a very small robot — a microbot — could use a shaped charge to “blow holes in people’s heads.” A microrifle, he said, could be used to “shoot their eyes out.”

I guess...I just don't get the prioritization of fears here...for decades we've also had the ability to kill each other with anthrax and sarin gas, things that can be dropped from large "delivery ships" and used to kill even more invisibly than these insect-sized drones. Why is it more likely that we're going to develop superintelligent autonomous insect drones than we are to annihilate ourselves with human-controlled mechanical systems, as we've had the capability to do so for many years now (nuclear ICBMs and so forth)?

View on HN

Drones have agency; they can target precisely and evade defense mechanisms, whereas gas spreads in well understood and constrained ways. You can't release a cloud of sarin or anthrax over Washington DC and instruct it to seek and destroy the entire high-level leadership of the US government. With drones, you can. And it doesn't require superintelligence, just face recognition at the level that already exists, and some engineering work to put all the pieces together.

Is a drone attack worse than a nuclear detonation or mass biological attack? Maybe not. But if you haven't noticed, people are scared shitless of nuke and bioweapon attacks, and with good reason. So it doesn't seem unreasonable to question whether we should be directing government resources towards developing weaponized drone technology that will inevitably be used against us.

Sarin gas doesn't decide deploy itself. A superintelligent autonomous insect drones might. That's the difference.

>human-controlled

They might not be human-controlled. That's why it's extra scary.

To reduce that a bit farther, it doesn't have to be desired control. Anomalous behavior in the form of bugs can have the same undesirable effects, and simpler systems have fewer bugs.

How many bugs are there in my 1960s vintage toaster? How many bugs are there in an ICBM's control circuitry? How many bugs are there in a modern kernel?

You make lethal devices "smarter", you'd better make sure you know what they do. This isn't impossible, but it's not easy or quick either (something which I think we can all agree drives a lot of systems design). Given that a lot of machine intelligence is predicated on statistical methods and eventual convergence... maybe not the best combination?

Yeah...I suppose my argument construes too much of a strawman. It doesn't have to be that we invent machine superintelligence...it's merely enough to naively believe in our automated, neural-networks systems that, without proper feedback controls, can cause catastrophic damage...whether they are sentiment in doing so is besides the point.

Interesting side point: I wonder if emergent-MI systems will be more resistant to attack?

From a biological standpoint, what we're basically doing with deploying code currently is creating and propagating generations of clones. Which, didn't work out so well for bananas...

"The single bug that causes all smart-fridges to murder their owners in a pique of crushed-ice-producing rage" would be less of a concern as we move towards more exogenous (with respect to the base code) processing systems.

Would you like to play a game?

> Why is it more likely that we're going to develop superintelligent autonomous insect drones than we are to annihilate ourselves with human-controlled mechanical systems, as we've had the capability to do so for many years now (nuclear ICBMs and so forth)?

Proliferation issues and barrier to entry.

They don't need to be super-intelligent if their task is just to find the nearest eyeballs and it's much much harder to stop someone producing killer insect drones - the industry for which has many uses - than it is to stop them producing an ICBM (the industry for which is fairly specific.)

We're talking about something someone could conceivably do, on a small scale, in their garage given a couple of years and very minor funding. From there it's just a matter of scale.

Reply on news.ycombinator.com