1,000 tech experts warn against AI arms race
bbc.co.uk
bbc.co.uk
>First we had the legs race. Then we had the arms race. Now we're going to have the brain race. And, if we're lucky, the final stage will be the human race.
>John Brunner, The Shockwave Rider (1975), Bk. 1, Ch. "The Number You Have Reached"
Great book, though.
What exactly would be banned?
I've been very critical of claims of the dangers of AI, but this isn't about AI - this is about giving lethal weapons to autonomous systems.
And that's just stupid. All software has bugs, and even if you don't have a problem with autonomous killing systems - I certainly do - it's incredibly irresponsible to build systems that could kill just because someone left out a semicolon or couldn't convert between miles and km.
It's taking the "blue screen of death" experience just a little too literally.
https://en.wikipedia.org/wiki/List_of_coups_d%27%C3%A9tat_an...
Usually if a leader is assassinated, the assassin is a member of the country's military.
The US is comparatively rare in having had no coups since independence and only one civil war. The US also has very strong indoctrination against this kind of thing, but individuals are just as vulnerable as robots to making the wrong decision if they are fed false information. Such as endless attacks on the legitimacy of the president's election, birth certificate, allegations of being a foreign agent, etc.
The US is of course not above assassinating foreign leaders, conspiring to have them deposed, or providing arms to military coups against democratically elected governments. Allende is probably the biggest example.
Edit: and of course there's always the risk of a single rogue individual, from Dr. Strangelove's " Brigadier General Jack D. Ripper" to the very real Ford Hood massacre. In fact, Dr. Strangelove should be required viewing on this subject; not only is it darkly hilarious, it's all about setting in motion destructive partly-automated systems which cannot be countermanded.
What happens when leaders know that, no matter what, never are going to be "targeted" by their own automatic military?
(Spoiler alert for those who haven't seen the first 3 seasons.)
The Samaritan works like you described - turns data and metadata into kill decisions. The Machine just outputs a social security number of a "person of interest". Figuring out why is the job of humans.
The show features an interesting discussion about what happens when an AI following the first method (data -> kill decision) starts lying to you to further its own agenda. The same issue applies when you have a black box of sub-AI complicated analysis algorithms that make the decision.
If the system is using all available classified information, then an explanation of its output would have to include the security classification of every input, and the system of classification may also be such that no one person is cleared to know everything. Therefore nobody can know the rationale.
(I've made this up, but I think it's the most plausible way such a thing would get built, and it's the logical outcome of "no read up/no write down" security rules.)
In terms of the real world, how many systems do explain step-by-step how they arrived to their conclusions? With some of the analysis and reasoning algorithms, it would be unfeasible to require humans to double-check every step. How would you quickly tell if a Bayesian network aggregating probabilities from hundreds of observations is correct?
We don't even require explaining themselves from humans. Instead, we have the concept of trust, which is something one earns. As a commander, I will trust that my subordinate is doing his analysis in good faith, but because there is abuse, we try to have an audit trail that can be used at later date to detect who acted wrongly, and should be deemed untrustworthy. Especially in combat situations, there is no time for an individual to evaluate every piece of evidence.
So the point is - are you willing to trust the algorithms you don't understand? With humans you at least share the common mind architecture. In the show, Machine was designed to avoid this question altogether by refusing to make decisions that carry moral weight, only to point out potential persons of interest for human agents to investigate.
But in the context of the script: the designer wanted it to be a 'black box' so nobody controlling the machine used it for it's owns goals. The machine only outputs the minimum relevant information. It's not a bad show.
In wars, invasions, etc, including those done by "democratic powers" few care about "misindentification" (or even far worse offenses to the enemies including civilians). That's why the mostly go unpunished or just get a slap on the wrist.
"Kill the all and let god sort them out" is the modus operandi.
The alternative is indiscriminate death that we see in mass bombings, mine fields, artillery strikes, and drone strikes.
The problem is that those intelligent systems will be overlorded by the same people overlording the actual military.
At least in our current situation a human has to pull the trigger and that person could be potentially charged for crimes against humanity which would make it thinking it twice and potentially resist to follow a direct order. You won't have that with a machine.
An AI can make split second decisions without panicking and without any consideration for its own "survival". For any mistakes an AI makes, those higher up will get the blame (ideally), not some 20 year old who was scared for his own life when he made the decision to kill everyone on that overloaded truck because he it wouldn't stop 10 seconds earlier.
That said, I shudder at the thought of a world in which people get killed by machines who will never be whistleblowers, who do not go home with post traumatic stress disorder, telling everyone how horrible war really is.
Tha't ok, but I think you are assuming that the "good guys" are the only ones that would have access to this. Truth is that once one government start doing this, all the others will follow and even the organized crime would have access to that technology.
You are thinking in Gov. A vs. Gov. B both using machines. My concern is that after that, Gov. B will use machines against humans C inside the same country, or in the neightborhood. And in those cases, "technical errors" can be used as excuse after a tragedy. And that's only one of my concerns... add organized crime and terrorism to the mix and you will have a very explosive soup.
If we can make humans do it, I suspect whatever is controlling future craft would do it too, especially if programming is much easier than it is to program humans to be murderers.
(a) kill unfriendlies (b) save neutral lives (c) save friendlies (d) keep minimal costs (e) thoroughly destroy opponents means of warfare
Given that the people setting priorities already don't keep (b) or (e) as it relates to civilian property in mind, how likely is a system to come into existence trained to consider (b) or (e) highly enough to say, not make the assertion that any amount of foreign destruction is okay if it is likely to minimize costs in the next 3 months? (i.e. We won't have to sweep that building if it's leveled.)
Another alternative to indiscriminate death is calculated efficiency driven by algorithms tuned to save US money, kill their guys, save our guys, and if possible while doing the other three, don't kill neutrals, friendlies, or neutral property.
And to wrap it all up, we'd have to stand by and watch as a medieval death cult like Islamic State overruns an entire region.
War is very rarely the right solution. Arguably, wars had a large part in creating IS in the first place, and I was very much against most recent wars. But for reasonably enlightened democratic nations not to have the capability to win a war if need be, ultimately means to be at the mercy of any insane barbaric ideology or religion that comes along.
The criminal in this scenario is the government.
In other words, should we not build intelligent robots because someone might give them a gun to make them killer robots?
If it doesn't take expertise to train the AI though, the goose was cooked a long time ago.
Do they have to be killed though? If we have machine soldiers and the enemy combatants aren't an immediate threat to actual humans shouldn't we be trying to detain them instead?
No doubt there will always be a Syria or North Korea that uses AI-controlled weapons anyway, but I think civilised countries could avoid them, at least while we have the military advantage.
You could argue that a state would own an AI and just not use it to direct attaks.
Meanwhile, few warn against widespread surveillance, the repurcursions from the use of drones, etc.
> Meanwhile, few warn against widespread surveillance, the repurcursions from the use of drones, etc.
The repercussions from drones could become worse if they are upgraded to include entirely autonomous AI, where there is no human involved in the "should I kill this person?" decision.
That and media have never ever exaggerated anything they printed.
Now, I happen to respect both men as actors/directors. I also see they have both participated in scientific documentaries (as narrators/hosts, I suppose). I could understand if they were part of a "public relations board", which usually requires some degree of celebrity power, but a scientific advisory board? Doesn't this require one to actually be a scientist or a subject matter expert?
Hanlon's razor. Not everything is a media conspiracy.
If you think about it, the US has not even ratified the Nuclear weapons testing ban treaty. I doubt it will ever consider an AI weapons ban.
- weak non-self-replicating;
- weak self-replicating;
- strong.
Besides, maybe it's an egoistic point of view. Many species have perished while humanity established itself. Does it really matter if we go extinct, if the result is going to be a superior species?
In general, not wanting to die or go extinct is not considered an "egoistic point of view".
Or, let's put it this way, from all the egoistic points of view, it's the most excusable.
Why the duck should we care for a "superior species" (and a mchanical one at that)?
Would you let a "superior country" fuck up your own country?
Would you let a "superior person" kill your family and use your resources to sustain themselves?
Is it OK to extinct dolphins and lions, since we are a "superior species"?
Even more so since "superior" in this context has nothing to do with "morally better" but just means "more powerful" and "more fit to overtake others and survive".
http://www.supportkillerrobots.org/
Shameless plug, I wrote it on a lark.