Hacker News Reader

Comment by cromwellian | Hacker News Reader

The near term danger of AI isn't in hyperintelligent SkyNet like systems, it's in human controlled autonomous weapon systems and "stupid AI"

What you should be fearing is military drones being given the ability to make decisions on targets or to fire, even with human assistance, and these systems won't just reside in the hands of large governments either.

Already police and militaries around the world are using abstracted forms of force, wherein target are identified with algorithms, and then force trained on those targets

What's do you think is going to happen first, SkyNet, or a predator imaging drone telling a human operator falsely that the current image is a terrorist?

What's going to happen first? SkyNet, or self driving cars putting millions of people out of jobs because of a lack of demand for drivers in transportation, or manufacture of cars? (I'm not saying it's a bad thing, but it will be very very disruptive)

If SkyNet is a threat, it's 50 or 100 years off I think. "AI" as it is now, is no where near the capability people are talking about. It's sheer hyperbole.

View on HN

    > a predator imaging drone telling a human
    > operator falsely that the current image
    > is a terrorist?

I think you're missing how this can be a really good thing. Right now this is happening but the "algorithm" is some unpredictable twenty year old in uniform.

Are we going to have hearings auditing what the algorithm is like? is there going to be a scandal because isBrownPerson() is discovered in there somewhere? Is someone going to run it against a video collection of regular daily life and discover the drones would be indiscriminately bombing innocent situations if given the chance?

The end result is that we're going to have to effectively have to come up with rules of engagement for every scenario. I think this will lead to war being fought more justly and with less collateral damage than ever before.

The fear is that instead of an algorithm and a human double checking each other and refining each other that they will simply add. So we end up with the worst decisions of both human and computer.

I hadn't considered that angle before, there needs to be a lot of oversight (in general when we're killing people, not just computers) but I think you're right.

It's adorable how you think it will improve how war is fought. These autonomous systems will always have a manual override and I will never believe that the review process will be given proper oversight. The armed forces will always err on the side of destruction in any potentially vaguely violent scenario. Fear and War is too prosperous to be peaceful.

If this is true, one wonders why they waste all that time writing rules of engagement, and prosecuting those who don't follow them.

We generally only prosecute those who break those "rules of engagement" when they are not ourselves.

This cannot ever be a good thing; drones justify terrorism because it's the only available response to fighters from the "other side".

Thinking we can wage wars without risk to ourselves or our "soldiers" is preposterous.

"Just" wars are a ridiculous idea rooted in superstition and hubris.

    > drones justify terrorism because it's the only available
    > response to fighters from the "other side".

We had similar terrorist attacks before we had drones, read what the likes of Bin Laden said about why 9/11 happened. It's because the infidels are on their homeland, not because they're using some specific technology to attack them.

More generally people resort to terrorism in the face of an overwhelming enemy. The west could relatively easily win the wars they're fighting now in the middle-east with their WWI armies if they were willing to care less about civilian casualties, that's how lopsided the odds are.

    > Thinking we can wage wars without risk to ourselves or our
    > "soldiers" is preposterous.

Drones as a technology would happen regardless of whether or not the armies developing them were concerned about their own soldiers. They're simply the logical next step in technology, e.g. fighter jet development has stalled to some extent not because the technology has reached its limits, but because if you pull more G's than you do now the pilots might die.

1. The current algorithm might be an unpredictable twenty your old in uniform, but there is accountability for their actions: it is somebody's actions. Who is responsible for the illegal actions of AI? You start to have to untangle racially motivated assault against "it shouldn't have done that".

2. People will not come up with "responsible" rules of engagement as you seem to suggest. isBrownPerson() will be coded into the algorithm but not in a discernible way. Sure, you could verify this against a video collection: like we all know the statistics of profiling and incarceration rates for different races.

I wonder whether "a human in the loop" is some kind of illusion; for longer than we have had recorded history, we have killed our neighbours because an authority have told us to. In the latter half of the 20th century we nearly perfected the conditioning needed to make soldiers shoot to kill when told to. I wonder whether it will make a difference if the order is given by a human a or an AI. And if the soldier is going to shoot to kill regardless, why not just have an AI do that as well.

Yeah that is an interesting paradox. It feels wrong for an algorithm to decide whether to kill a human, but what is so much better about that decision being made by a human?

If that seems extreme, consider the self-driving car problem. 30,000 people in the US alone are killed per year in car accidents involving human drivers. What is the acceptable number of motor vehicle related deaths per year when all cars are self-driving?

>Yeah that is an interesting paradox. It feels wrong for an algorithm to decide whether to kill a human, but what is so much better about that decision being made by a human?

Obviously because those humans then share and bear the moral dilemma of doing it or not. Which also affects the outcome, as opposed to just being some inconsequential detail. Humans can demand out of an injust war (as they've done time and again). Killing machines can not.

It's the difference between blindly following orders (which not even all Nazi's did) and being able to have a change of heart.

It's also the sheer (number of magnitudes) more killing efficiency of the killing algorithms, when a machine could kill 10000 people in the time it takes for a single human to make a decision about 1 of them -- until we find there's a != when we needed == in the algorithm.

as another commenter noted there are currently physical limitations that would not allow e.g. drones to kill 10000 people in an instant. I don't think for example nukes will ever be under the control of AI algorithms ever...

    > 30,000 people in the US alone are killed per year in car
    > accidents involving human drivers. What is the acceptable number
    > of motor vehicle related deaths per year when all cars are
    > self-driving?

The right answer to that question is "29,999 and getting better", but unfortunately luddites abound even in modern hi-tech society.

Well, we'll still have Hitlers (or benevolent Trumans deciding to drop on Hiroshima etc). Those would be the ones building, enabling and deploying the combat AI.

What we won't have would be Vasili Arkhipovs, Oskar Schindlers and Chris Taylors...

> In the latter half of the 20th century

This is an odd statement. In my reading of historical battles, I've never seen a mention of the idea that soldiers before 1950 did not shoot to kill when ordered to.

I've seen this figure before, and read that they changed firing range practices after the war to make killing more automatic. This was in Gwynne Dyer's "War", and he's normally reliable.

However, the original statistic is apparently from a not so credible source, a study by SLA Marshall in WWII that's not so well regarded. This thread goes into some depth:

http://msgboard.snopes.com/cgi-bin/ultimatebb.cgi?/ubb/get_t...

Not sure what the truth of it is.

Another more realistic potential threat is police relying too much on automated crime prediction. They are already embracing it:

http://www.firstcoastnews.com/story/news/crime/2015/10/19/po...

An old thread on the topic:

https://news.ycombinator.com/item?id=4185684

> relying too much on automated crime prediction

Including many variations of that same problem. "Sorry, you don't qualify for ____ because our 'clever' assessment algorithm doesn't account for your particular situation." Humans with decision making power correct these kind of problems all the time.

Computers with clever algorithms can be incredibly useful, but they are still just tools. Unfortunately, given how bad we humans are with statistics, it's easy to overestimate[1] their accuracy.

In addition to the raw accuracy problems, there is a risk of prejudices being baked into algorithms. We already see this with "redlining" and other housing practices, where loans availability ends up being coded racism. With the added complexity of machine learning and other prediction and analysis methods, it is probably a lot easier to hide improper discrimination.

[1] reason #57 why statistics should be a mandatory HS math class, right after algebra

It's a hairy problem. Not understanding statistics goes both ways - and from what I've seen, it usually leads to overestimating the value of human input. People are quick to distrust or reject machine's judgment when it disagrees with their biases and generally, it's much easier to debias a machine (or code it properly in the first place) than to debias humans.

While I agree that humans may enshrine their prejudices in code which will later will turn hard to adjust, again what I fear more is a reverse scenario. A machine can be perfectly fair and people won't like it, because it fails to apply the biases they want. People often apply counter-discrimination to compensate for what they believe is unfair treatment.

A mandatory class in statistics and applied probability theory could maybe help the next generation to accept that what is fair may not look so at a first glance.

> A mandatory class in statistics and applied probability theory could maybe help the next generation to accept that what is fair may not look so at a first glance.

To be honest, I don't think this will help much since most people won't interested in it and will forget what they learn as soon as the class is over. Also, being an expert in statistics can help you better understand reality, or it can help you present data in a way that confirms your biases (even unintentionally). It's sort of like sophistry, except not quite to that degree or moral ambivalence.

edit: If you want a good example of this, see Scott Alexander show how parapsychology can actually come up with some pretty good statistics while appearing to check off all the right boxes with regard to experimental design: http://slatestarcodex.com/2014/04/28/the-control-group-is-ou...

There was a good piece about this on the radio today.

http://www.kcrw.com/news-culture/shows/to-the-point/can-big-...

Luckily there are physical limitations. There are practically two ways to arm a drone. Bombs and guns.

A small flying drone operating an 9mm pistol has extremely poor accuracy. Especially with multiple shots. If you increase the accuracy, you have to increase the mass. Which makes it bigger target. Currently regular guy with little training and a rifle can probably shoot down a gun wielding drone most of the time before the opposite can happen.

https://www.youtube.com/watch?v=xqHrTtvFFIs

Then there are drones with bombs attached to them. One of the most sophisticated is called Hellfire missile. But it's needlessly big and expensive. The quad copter in next video costs 100$ and has payload to carry regular hand grenade.

https://www.youtube.com/watch?v=Lb2Tpp3CIoY

You could go RC car route. But you again need a bit of size to conquer stairs. Also it's easier to track where it comes & goes.

https://www.youtube.com/watch?v=kUXRMDK3r7s

I'm less worried about government doing bad shit and more worried about private citizens getting nasty. Autonomous flight, GPS navigation, address from google maps and bit of facial recognition. First civilian drone murder is just matter of time. People kill with more ease the further they can be when it's done.

I'm guessing this internet privacy thing will seem like child's play when all of sudden everybody wants to hide their physical address. Once that is handled, it's only bit like having rabid dogs with homing device.

"Luckily there are physical limitations.... I'm less worried about government doing bad shit and more worried about private citizens getting nasty."

The MSF Trauma Hospital in Kunduz, Afghanistan (3rd October) didn't have this luxury and in the fog of war, [0] 12 MSF medical staff and ten patients were killed. [1],[2]

Killing by remote sensing, is indiscriminate. Adding AI is another order of stupidity.

References:

[0] http://arstechnica.com/information-technology/2015/11/how-te...

[1] http://www.msf.org.uk/article/in-memoriam-msf-colleagues-kil...

[2] http://www.theguardian.com/us-news/2015/oct/06/doctors-witho...

Not sure that it matters for the sake of argument, but the weapon system used in question was a human-piloted, human-manned AC-130 gunship, not a drone / remote platform

"the weapon system used in question was a human-piloted, human-manned AC-130 gunship"

True, read the Ars article: "The BMC crew is responsible for steering the aircraft to targets, identifying them, and shooting them; the aircraft's battery is slaved to the sensor suite for targeting." - Aircrew, well trained, professional. As ethical as you can get.

Now what happened is "A US special operations team on the ground, given coordinates of the Afghani NDS building by the Afghan forces they were working with, passed them to the AC-130. But when the AC-130 crew punched the grid coordinates into their targeting system, it aimed at an open field 300 meters away from the actual target. Working from a rough description of the building provided from the ground, the sensor operators found a building close to the field that they believed was the target. Tragically, it was actually the hospital."

Will adding AI to this situation make any improvement when the real issue is ' lack of a "common operating picture,"'?

No, you're right. The only thing that may have happened was lack of connectivity to COP and no person in the loop to authorize, resulting in a stand-down of operation and no ordnance on target. Great if it's a hospital, not so great if it's actually a bunch of enemy that you need to remove to save your soldiers on the ground. Tough, tough thing to get right all the time. I think a measured, hybrid approach will probably get you the best outcome but I'm no expert in this field.

just gets muddier and muddier ~ https://www.washingtonpost.com/news/checkpoint/wp/2015/12/11...

Total aside, but the United States should really promote some sort of beacon / FLIR-readable indicator for hospitals. Their existence would have to be kept a quiet so there weren't a bunch of false positives floating around, but I would think the cost is minimal for the returned value.

AI can probably actually improve this particular incident. Having a "rocket" that doesn't explode but drops an AI controlled drone that kills (or even just disables) exactly who is targeted seems like it would be far superior from both the military perspective and the moral perspective.

This sort of thing: https://youtu.be/DTqa-NEwUbs?t=94

It could even drop "phone home" over radio, allow the operators to point out the targets at that point, and then fire.

We've been hearing about "smart weapons" since the early 90s and they don't seem to be getting any more intelligent as time has gone on.

Governments already have their AI killer robots. It's called "military personnel" and "policemen". The increase of government lethality towards regular people should be somewhat minimal compared to current tech. This has to be controlled politically, hence democracy.

But governments could greatly increase their capability for surveillance.

    > Adding AI is another order
    > of stupidity.

That is predicated on the assumption that AI would make worse decisions than humans, which I'm really not sure is a given.

In Prof. Tegmark’s presentation at the UN he mentioned the possibility of extremely cheap drones that approach the victim's face very quickly and pierce a bolt into the victim's brain through one of their eyes. That wouldn't require a high precision projectile so it will be easy and cheap to build.

Are you suggesting there is something about drones that makes them fundamentally inaccurate? It would seem to be that advancements in computer vision will solve that problem in short order. Particularly with the VR industry introducing strong incentives for marginal improvements on image-based tracking.

Newton's third law; Hence the comment about drone size/weight and mentioning the small caliber weapon it would have to use.

It could expand out a big frill along a plane perpendicular to the shot when firing to transfer the kickback momentum to the air.

We already drop quarter of a million dollar bombs to kill single targets. Having a drone which turns into a disposable rocket to hit a single target is hardly unfeasible.

They are still going to optimize everything, something that can fire X number of shots instead of being a one-off rocket will stretch their budget farther, etc.

Right, but that wasn't the point - the point was that thusfar, the appeals to mechanical limitations of drones as weapons of war have been based on very limited thinking.

Umm, it doesn't have to have good aim if it can get close enough, eg. a drone small enough to crawl into a person's ear and then fire/detonate.

How big of an explosion do you think a drone small enough to fly through your ear canal is really going to be? How does it even reach its target with such a minuscule mass? The target exhaling would blow it away (and likely destroy it)

obviously enough to blow your brains out once it's in your ear

Obviously it would require enough to blow out your brains to be effect. How do you propose to create a drone so tiny and light to also carry the payload required to do that? Just because the payload itself could fit into someones ear doesn't mean the delivery mechanism can, well, actually deliver it

Miniaturization is expensive. And you need some way to get to the victim that doesn't take years. This increases the cost too. Most people would be just "too cheap to kill".

> If you increase the accuracy, you have to increase the mass. Which makes it bigger target.

Boeing is developing compact portable laser weapons technology that could conceivably be mounted on drones.

http://www.livescience.com/52121-laser-weapon-destroys-test-...

The laser itself might be compact enough. But---where do you get the energy from?

I'd like to point out, that 2 kilowatts is the power AK47 can give as output of kinetic energy when fired at full auto.

If you look past the hype, lasers are far from ready.

You'd probably really like the book The Future of Violence by Benjamin Wittes and Gabriella Blum

I think the actual threat from AI is far more pedestrian than "Skynet" or killer robots. The real threat comes NOT from AI itself but from people who will be able to afford to exploit AI for profit at the expense of increasingly large numbers of people who just become redundant. The rich will get fabulously rich while everyone else just becomes marginalized into serfs.

Another scenario that is less likely but still more likely than killer robots would be AI that simply loses interest in human affairs, stops interacting with us, and does its own thing-- like depicted in the movie "Her" !

The other problem is if AI can be programmed to commit physical crimes then only a select priesthood of vetted developers will be allowed to program them at the service of a concentrated elite.

Bravo to point out the obvious.

Politics, moral only exist because there a large quantity of human.

All AI researchers I spoke to laughed at this "SkyNet" like fantasies. According to them the reason why some people peddle these sort of horror stories is because they seek to get more funding by exaggerating the achievements of their field.

Snowden like episodes have shown us that we should not trust our government. The government does blatantly illegal things, lies when confronted and in many cases kills or jails our own citizens just because they are exposing the government. I am more scared of government using this AI to oppress its own people.

Imagine this, a person who is 18 years today becomes a potential presidential candidate or major activist in future. Some clerk at DC would simply run a computer program that will bring up the fact that this guy sent a sext to his 17 year old girlfriend in past, smoked weed when 19, bought beer when he was 20 etc. Such government would them implicate him in various court cases and jail him or even worse simply blackmail him to shut up.

Let's not overblow the threat. W was known to have done cocaine in his youth, and he was still elected, and I would expect statute of limitations to apply.

I am not over-blowing the threat. We would not know if government simply blackmailed people using such evidence. Many times you don't have to convict a person for the crime merely spreading a credible lie in media is enough. Obama has already used IRS to harass conservatives and we know what watergate was all about.

W was the supported choice of the power elite. It would go different Kshama Sawant was polling high numbers and such photos came out.

"If SkyNet is a threat, it's 50 or 100 years off I think."

Well we work on global warming even though the worst effects are likely decades off. I think one computer science professor put it something like this: "If experts told us aliens were likely arriving on Earth in 50 or 100 years, it'd make sense to start preparing now." You don't want to wait until it's too late to do whatever needs to be done.

>predator imaging drone telling a human operator falsely that the current image is a terrorist

Though that stuff isn't nice it's probably an improvement on previous technologies - looking out of the bomber and thinking that looks like some good buildings to bomb and similar.

Yeah, or looking at the monitor with the drone camera feed and thinking "hmmm, looks terrorist to me".

Well with AI they could do face recognition against their Twitter account and see if they'd posted any jihadi stuff recently

Not to mention what is happening with the NSA, which is essentially judicial-hearing-by-algorithm.

I think self driving cars and carts will also kill almost all middlemen. Why go shopping for food if you can get everything shipped to your fridge for 20-30% less, direct from the producer. Google will probably also offer to drive you for free to a medical specialist and start doing tests after you entered the car. So we probably need a lot less physicians too.

I'd argue that we'd probably need a lot more. Diagnoses would be more accurate so people would be inclined to use the health system more. Health work would be cheaper because the expensive doctor component would be used less. Time to go to the doctor could be more efficient (still working through the car ride) so people are even more likely to go. Overall that could likely mean higher demand for healthcare, including the human component.

Your food shopping point still needs middlemen. You have "chain of trust" issues with the producers. How do you know that you'll get good oranges from the orange grove. Who capitalizes the distribution system and designs the business model.

Considering historical examples, we'll see more use for people in the economic problems that are still machine hard. Machine easy economic problems will converge to resource costs + any premium for monopoly. This will free up more demand for other products. People still want to buy an easier, better life.

There are often 3+ intermediaries between many farm products and the consumer. Dropping that to 1 chould be a huge change. Cow, wholesale, bottler, distributer, store, you. Granted, there is a lot of variety in this as some farms do there own botteling etc, but some products pass through even more hoops.

There is also the pasteurizing step to consider, and plenty of countries have 'milk boards' (cartels by any other name).

So you think that when we finally have smarter than human AI, it's just going to make completely objective decisions?

First off, we'll never let such an AI to be completely objective, because, well, the paperclip scenario. And second, the AI will be about as "objective" as Google search is - in other words, not really. At the end of the day it's still humans that decide the algorithms for Google search, and it will be humans that decide the algorithms for the "smarter than human" AI.

Even without the hardware, I'm a bit surprised we haven't seen more AI-based approaches to finding and exploiting security vulnerabilities. I guess it's still easy enough that they don't need fancier tools; a well-designed fuzzer can do a lot.

Although I hear about machine learning-based approaches to fuzzers more and more lately. So I think you will soon!

Stupid AI in healthcare is a drastically greater threat than human controlled autonomous weapons will ever be, unless you're talking about WMDs.

It's not a close risk comparison. The world is far less dangerous today than it has ever been, that includes both terrorism and war. AI and autonomous weapons systems are not going to suddenly make the developed countries that deploy those weapons want to slaughter each other.

How many people die from medical mistakes per year in the US? More than died in all wars and all acts of terrorism combined globally in 2015. The AI that is no doubt going to show up throughout healthcare over the coming 30 or 40 years, will probably improve on the rate of human mistakes - while it still kills tens of thousands per year through mistakes in just the US.

Not that many people die from medicla mistakes. The hysterical oft quoted paper blamed every suboptimal outcome from a "medical mistake", ignoring the contribution of whatever cause put them in medical care in the first place.

with regards to spotting and identification I can see a system like what was shown in the last Dredd to come about, drones in the air and cameras feeding a large centralized room staffed by people directing others.

While the military can get away with remote action it won't happen with police in Democratic countries except possibly with non lethal force. I could see remote shut down of cars, even a flash bang or sonic system.

The real danger from AI isn't it taking over, it is from too many people checking out and not participating in life anymore

Reply on news.ycombinator.com