Drones may have attacked humans fully autonomously for the first time
nypost.com
nypost.com
Drones have several layers of added "decision making" that are out of human control making them far more dangerous. They get to autonomously decide, target, and kill based on many factors, using algorithms that we cannot trust or predict 100%. They also does it from afar. I know you can't trust a human 100% either but it's still the best we've got.
It's the difference between walking in a bog and encountering a lion. Both might kill you but they're in no way similar even in principle.
Autonomous killer drones are just automating murder and terrorism. Taking horrific weapons and making them worse by giving them better range, better targeting, and a brain that's smart enough to kill but stupid enough to do it as dangerously as possible.
This is the terror element. People will have to worry about getting blown up anytime, anywhere. That's why we keep making weapons with more range and better targeting, because waiting for the target to come to the weapon just wasn't horrific or effective enough.
Sure, landmines are really bad but we're inching closer to things that surpass landmine horrors in every way and makes them seem like the lesser evil. Because if there's one thing that's bad, it's taking an already bad thing and augmenting it. And that should really scare you.
If someone really wants a hit on joe random citizen, why would they choose one of these drones over the traditional method? Much harder for a weaponized drone hit to look accidental.
> why would they choose one of these drones over the traditional method
Why does anyone use automation or AI for anything? To have dozens or even hundreds of drones constantly in the air with close to no supervision from drone operators. When you're going after an enemy false positives and collateral damage matter very little, if at all.
> Much harder for a weaponized drone hit to look accidental
Nobody bothers making deaths look accidental in the places these drones will be used. If reality showed anything it's that they barely bother to argue when they bomb schools and hospitals beyond the boilerplate "we're fighting terrorism".
This kind of tech makes a bad situation even worse for many people in some parts of the world. But you'll start worrying about it only if/when someone decides to use it somewhere in a civilized country, closer to home. I'm sure so many people will suddenly find plenty of arguments against the autonomous killing machines then.
Drones have been used quite successfully for many decades now and clearly haven't developed any countermeasures for what most of their targets are concerned, "quickly" or otherwise. This is why we still read about drones bombing people out of existence, whether the correct target or not.
An autonomous drone just outsources the usual killing from the human operator to a script, making it even more likely to make a mistake and also apply the desired effect: terror.
I guess missiles are given an order for a specific aircraft.
In nuclear weapons it’s the difficulty in sourcing the nuclear material that prevents widespread proliferation. Imagine if the core could be made of something like iron, everyone would have nukes and our civilization would not survive.
The line that has never yet been crossed is the invention of weapons that decide when to launch themselves, at least in ranged indirect fire. Things like CIWS are arguably both autonomous and self-initiate firing.
Coupled with its indiscriminate nature, and denial of swaths of lands, and it's an cruel atrocity of a weapon.
Automated weapons I feel fall under something else.
Quadcopters are not far from having all of these qualities either.
I don't like this "whataboutism".
In addition, while landmines are terrifying and deadly, they stay in one place. People generally don't need to fear them when going about their day. Drones on the other hand are mobile, they could fly into a city and kill people. They could fly on your property and kill you.
The point is that they are entirely different concepts, and since this a discussion about drones, landmines have about much place in this thread as Tse-Tse flies, SUVs and low-head dams.
The concept of killing ai is not well defined.
There was a video a couple of years back of a kid in Texas attaching a pistol to a drone and firing off a couple of shots. Stuff like this is not rare or inaccessible, and it's only a matter of time before it becomes widespread.
It's easy to say "Well, the other side are bad dudes who have it coming", but even if they are genuinely bad dudes (and not just politically inconvenient) there's no war without collateral damage (a term invented by the US during Vietnam, when they were doing a lot of it).
It's not so much software that's the issue, it's people taking existing tech and weaponizing it.
Software has been in weapon systems since the 1970's from guidance to target acquisition. It's already there, and has been for a while. Machine learning is just another iteration on what's already out there.
The world knows how to kill en masse for a long time now with NBC (Nuclear/Biological/Chemical) weapons -- and software wasn't a part of Fat Man or Little Boy.
But consider the challenges the US had in the middle east for the past two decades.
* Consumer drones dropping hand grenades.
* IED's using manual triggers.
* Trucks loaded with explosives blitzing checkpoints.
* Cars with explosives driving up next to convoys.
* Land mines.
You don't really need custom software, just take existing tech and get creative with it.
Obviously correct, so in a sense the parent commenters concerns are not about something new.
But historically these weapons systems are massively expensive custom systems developed by defence contractors for major states. Costs, budget cycles, and technical limitations went some way to limiting their ubiquity and the rate they could gain functions.
So cruse missiles had autonomous terrain-matching terminal guidance, but the US needed the combined abilities of the NRO and General Dynamics / McDonnell Douglas / Raytheon etc to build it.
Whereas now we have free and triviably cloneable software (embedded OSs, drone flight and guidance s/w, ML and image recognition stacks, global mapping data) and a massive range (compared to event ten years ago) of dirt-cheap cots hardware (GPS, lidar, ...)for making autonomous moving objects. And often those devices are fairly thin hardware shells around yet more software.
The network effects driving these capabilities, particularly in software, and their seemingly exponential increase, are what I took to be the source of the parent commenters concerns.
If you look at the current tactics of insurgency, software is not a part of it. Enemies of the US (Al Qaeda, etc) already had tremendous success with the current insurgency tactics (IEDs, etc) which were low tech solutions, and were vastly hard for the US to overcome.
Why would they go to software when there's plenty of tried and tested low-tech solutions already to kill your enemies?
To kill more people, whilst taking even lower risks themselves.
They probably wouldn't write the software themselves -- they'd get it from e.g. some supportive dictatorship
(With "them", I'm personally not thinking about the talibans, but other groups)
Sarin gas? Nuclear weapons? Truck Bombs? Strapping bombs onto children?
Cheaper and more accessible though. Not many people can build or buy their own nukes or sarin gas.
People have written software to target buildings, soldiers, tanks, planes, missiles, etc. This is done via radar, heat signatures, cameras -- and software already. It's not such a big change really.
I believe the thing you really have to worry about is the proliferation of NBC. It's easier to drop a nerve agent in a war zone than it is to figure out how to make an explody drone to kill your enemies and not you, say.
Sure someone might have had the ability to make their own gun at home, but today there are far more people who are able to build lethal weaponry from consumer-grade materials.
It's a frightening development.
Consider the hobby of FPV drone racing. The designs and plans are widely available, and you can get a top-notch drone for about $2000 for a first-time purchase (less for the subsequent ones, since you'll already have goggles/radios). These drones are regularly built by teenagers and it's not difficult at all to add capabilities to the drones, such as target tracking, autonomous operation, software-defined radios for jamming resistance, etc.
> "The lethal autonomous weapons systems were programmed to attack targets without requiring data connectivity between the operator and the munition: in effect, a true 'fire, forget and find' capability," the report from the UN Security Council's Panel of Experts on Libya said.
I’m not sure why this is called “rogue” — apparently it was operating as intended
In this case, it seems the drone was supposed to do exactly that, and that's perhaps far more dangerous -- this is the first kill where the drone was approved to determine kill targets, and execute that decision, with no human to verify. This really shouldn't be a goal we have in the first place -- to remove humans from the decision loop -- and yet apparently here it is.
With this you’re creating a no-go zone for your enemies, and very likely safe for your allies (who you can tag properly with markers), but the difference between enemy and neutral is rather unclear (especially in the Middle East, where the line between combatant and civilian is very thin, almost indistinguishable even to humans on-site)
> One way of looking at human creativity is as a process of pulling balls out of a giant urn. The balls represent possible ideas, discoveries, technological inventions. Over the course of history, we have extracted a great many balls – mostly white (beneficial) but also various shades of gray (moderately harmful ones and mixed blessings). [..] What we haven’t extracted, so far, is a black ball: a technology that invariably or by default destroys the civilization that invents it. The reason is not that we have been particularly careful or wise in our technology policy. We have just been lucky.
Try to pull one out and you get a web of affordances and side effects and unforeseen interactions stuck to it
If we were to be wiped out by our technology I see no guarantee some extradimensional coroner would be able to point definitively to just one
Say it's climate heating, rapid-onset Venus mode. Was it the internal combustion engine? The coal plant? The automobile? The wheel? Language? Money? Industrialism? The limited liability corporation? How do you pick one?
No technological form exists in a vacuum, likewise Technology is neither a monolith nor a ball pit of functionally independent atoms
I agree the analogy isn't perfect, but one can say that despite technologies not having a discreet origin, we can reason about them in retrospect. So maybe modern artificial neural networks were initially made possible by improvements in microscopy and experiments in cell culture to deduce the mechanisms of biological neurons, but we can reason about ANNs as a technology that is changing what is possible by a machine.
I imagine Nick would also be inclined to be fatalistic about technological progress, in that these "black balls" will inevitably be found. His main stance is that we should be enacting the global ethical frameworks around these technologies much quicker.
Yes, agreed, good point
I also agree about Nick's probable fatalism, and I think his work is brilliant and valuable, however one thing irks me. He's made a successful career out of hypothetical "black ball" speculation while saying little about the potential for blackness of the tech we already have. He frames existential risk as if we are on the cusp of dramatic danger but still relatively safe. In truth we are already over the cliff & falling rapidly
Take his paperclip maximizer. They already exist! We need not wait for some future AI to realize a badly misaligned objective function. We're already determined to transmute the remains of our homeworld's long dead into plastic and atmospheric carbon
Also some balls (e.g. nuclear weapons, stray asteroids) make it a lot easier to point to something immediate and mostly unambiguous
That's a black ball.
A very popular story but it was also primarily 1 of the 8 scientists working on the Manhattan project and only because he hadn't had all the math showing how incredibly unlikely it would be shared with him. It was already a settled "No, not going to happen." before the first test was conducted. The story ends up being a popular soundbite but with no real teeth on it. [0]
[0] https://blogs.scientificamerican.com/cross-check/bethe-telle...
Oddly, this is kinda freeing in a nihilistic way. If you know that you and your species are going to die eventually (which you do, the eventual fate of the universe is heat death), you can focus on living for the present instead of brooding on the inevitable death of mankind, and just be grateful that you and your species happen to be alive at this particular time on this particular planet where life is actually a thing.
Space is big. Earth is old. When all is told, we just don't know.
"THEY sent A SLAMHOUND on Turner’s trail in New Delhi, slotted it to his pheromones and the color of his hair. It caught up with him on a street called Chandni Chauk and came scrambling for his rented BMW through a forest of bare brown legs and pedicab tires. Its core was a kilogram of recrystallized hexogene and flaked TNT."
The clients were either the Libyan government (or defense contractor), or the the breakaway faction (or defense contractor), but we don't know which
We don't know who it attacked
We don't know if it killed them
That's frustrating.
Anyway no serious drone introspection is going to come until non-US drones attacks US assets, and ideally because they don't like a US ideology that there is no state-side consensus on changing
Very akin to bans on mustard gas or nuclear proliferation
You might believe in an omnipotent state that will close all the markets and go after node operators making them cease.
I believe that transactions will continue and drones will continue blowing up behind people's heads.
With drones!
I'd almost prefer that this showed something like someone's head getting blown off their body, to demonstrate the actual brutality of what it is designed to do, over making it seem like some video game.
They're a really twisted group of people that mostly feel like they have no other opportunities (that maintain their lifestyle). Not so different than people employed in any other industry, just important to understand.
If you’re wondering why it’s an effective video, it’s because it makes me want one. I don’t even have anything to blow up. That’s the essence of effective marketing.
We can also recognize that in the wrong hands, it’s an evil device. But so is a gun. And I would imagine the reactions to the first portable guns may have been similar.
Lots of people find guns cool, and I have lots of fond memories of shooting with my dad and a friend. Learned about gun safety and how to properly care for one. I also learned I wasn’t responsible enough to keep one, so I’ve never bought one.
This is obviously different. But it’s not inconceivable that in the future, you might be able to deploy a sentry quadcopter designed to chase down (not destroy) someone who just attacked you. I was shocked that there was a drive-by shooting in my neighborhood a few months ago. Would’ve totally sic’d a drone on them the moment I heard the gunshots.
Dunno. Sometimes I feel lonely for feeling this way, when the only acceptable social response is outrage or indignation. It’s cool to blow things up, and humans have had a long and proud tradition of making things go boom.
The key is to make sure that this tech won’t have unforeseen consequences. But as far as I can tell, the consequences appear to be foreseen in this case. It’s a drone that goes boom when next to someone.
All that said, I agree that it’s a terrible idea to let it decide to do this automatically. But in a warzone, it’s not impossible to imagine some heuristics that might make sense in certain highly constrained circumstances.
Of course, most dictators don’t really care about minimizing civilian casualties, and dictatorships seem like the target audience. But, ditto for guns.
I'm not sure what your point is here? The 'but' implies that we all now agree that guns are basically OK which is not universally true, and I would say increasingly the opposite.
Also your conception of 'the wrong hands' and other peoples' are not necessarily the same - one man's terrorist is another man's freedom fighter, and so on.
> But it’s not inconceivable that in the future, you might be able to deploy a sentry quadcopter designed to chase down (not destroy) someone who just attacked you. I was shocked that there was a drive-by shooting in my neighborhood a few months ago. Would’ve totally sic’d a drone on them the moment I heard the gunshots.
I think this is exactly what scares people - there are people who would sic killer drones on people Judge Dredd style, out of a misguided sense of moral duty; I heard about a recent case of someone using the 'stand your ground' laws to justify (and be celebrated for!) going to a neighbour's house during a robbery and shooting the thieves in the back, killing them both. Not to mention that in a future where these things are available, that drive-by would've instead been a fly-by killing and you might never know who did it.
You talk about these people like their lives are very important, and that it's both worthy and honorable to protect them.
In reality, this is what it's like: https://twitter.com/theshawwn/status/1301407648420593664
It was scary as hell to have a drive-by happen. The cops couldn't do anything. They were long gone by the time the police came. And then they came back and shot up the same place again a few hours later.
If I had a drone that could track them, they may have been brought to justice. I don't see this tech the same way you do. I see it as a stepping stone towards a world where people can protect themselves and their loved ones, as they do with guns.
I think we'll just need to agree to disagree. I have a lot of things I could say, but I just don't get how people can calmly live in a country that was earned through bloodshed, while decrying those who were willing to lay down their lives for you.
This is a military device. It's designed to kill people, because this is one of life's unfortunate necessities. I think the people here who are saying that there's nothing honorable about killing someone should teleport themselves back to Midway in 1942 and tell your great-grandparents that, face to face.
> You talk about these people like their lives are very important, and that it's both worthy and honorable to protect them.
> In reality, this is what it's like: https://twitter.com/theshawwn/status/1301407648420593664
> It was scary as hell to have a drive-by happen. The cops couldn't do anything. They were long gone by the time the police came. And then they came back and shot up the same place again a few hours later.
I don't personally care so much about the lives of people who engage in this kind of behaviour, but I do care about living in a peaceful society where violence is regulated by the law rather than being handed out vigilante-style. Arming everybody does not prevent gun violence, it enables it. You need only look at the blatant correlation between the number of guns per capita in the US and the number of shootings in the US. I'm in the UK - you know what we don't have lots of? Guns. You know what we also don't have lots of? Gun violence. It still happens, of course, but it's far rarer, and I don't feel the need to arm myself just to stay safe. That drive-by you were unfortunately so close to was made easier by increased access to weapons.
> If I had a drone that could track them, they may have been brought to justice. I don't see this tech the same way you do. I see it as a stepping stone towards a world where people can protect themselves and their loved ones, as they do with guns.
The trouble is, the easier it is for you to get hold of this kind of drone tech, the easier it is for everyone, which obviously includes those who would do you harm. See my comment above. You want a drone that can follow people around? So do a host of spurned lovers and over-protective parents and so on and so on - it's not the clear win for society that you seem to think it is.
> ...I just don't get how people can calmly live in a country that was earned through bloodshed, while decrying those who were willing to lay down their lives for you.
You seem to think I'm ragging on the military, but I'm not - perhaps you're referring to other comments? In general, one's neighbours shouldn't be laying down their lives for each other on a day to day basis unless they're in the military or the police, who are, at least in theory, trained and regulated in their use of force and when to deploy it (yes there are obvious potential problems there too, but that's a different discussion).
I would also like to point out that the bloodshed is supposed to be temporary - once the wars are won and safety is assured (I'll assume you're referring to the World Wars here), we're supposed to stop shooting each other.
> This is a military device.
So are AR-15s - how many of those are in the hands of twitchy civilian militia groups and other randomers with a grudge?
> I think the people here who are saying that there's nothing honorable about killing someone should teleport themselves back to Midway in 1942 and tell your great-grandparents that, face to face.
I'm willing to bet that the majority of those who served in World War II didn't see anything honourable about it either, just that it was a grim necessity, which it absolutely was. Honour and glory are concepts the armed forces use to encourage people to do necessary but horrible things - we should of course be eternally grateful for and respectful of their sacrifices, but I think it's actually disrespectful to the fallen to glorify war, as they gave everything so their descendents wouldn't need to engage in it.
In this case, you feel that war is a grim necessity, and also admit that this is a military device. If your country is in a war, wouldn’t you want to be on the winning side? Culture often doesn’t survive losing a war.
If so, then I don’t understand why people are against development of this tech. If you’re on the battlefield, you bet that you’d want a way to take out the people shooting at you from the hill on the other side. Usually that means calling in air support. Here, your air support can be almost pocket sized.
It’s pretty much as simple as that, and I think the crux of the disagreements. People seem worried that this tech will get out of hand. I am too. But it also feels inevitable.
Would you want to fight a war without guns? Because it sounds similar to an argument against developing guns.
What happens when people who did the driveby get the technology (if you can buy the drones so can the bad guys).
What happens when your autonomous drone decides to shoots neigburs kid for playing hid and seek, in a fashion that drone algorithms find objectionable.
And suppose your drone goes after them, finds them, then what ? You do realize that after, the drive by is done and they "flee" the scene, you can't just go after them and shoot them as a civilian? You can defend your property (someplaces ) but that does not include chasing after them.
Is the drone smart enough not to shoot when there are other people behind it's targets ? what if it's Halloween ?
Supose those drones get sold, and given current IOT security standards, someone hacks them ?
Certainly long, but I'm not particularly proud of the many ways we've found to kill each other.
Tripwires attached to a claymore fall under boobytrapping laws under arguments of reckless endangerment. I've heard it reasoned as there's no mind behind the assault and so it's reckless even if it harms someone breaking and entering.
Facial recognition gets closer to an intelligence behind the weapon, but a less than human intelligence (less than canine?). At what point does your "tripwire" have enough intelligence to be legitimate under castle doctrine?
It also kind of has to explode itself to do it’s job, which feels like a waste to me. So many things you can do with a drone that are immediately useless if it blows itself up.
This scares the shit out of me. Call me naive but the idea that somebody on the other side of the planet can set me as a target in a device that they just set and release is completely inhuman. If you want me dead, then do me the curtesy of at least putting your own arse on the line to achieve it.
The potential for this type of device to be cobbled together is well within the capability of pretty much any reasonably funded electronics hobbyist, and I think one of the reasons that we haven't seen it more frequently is that hitherto it's not been morally acceptable. If this type of weapon becomes mainstream in government arsenals then you can kiss that goodbye.
I don’t like it either, for what it’s worth.
The immediate second order effect is, people start buying drones programmed to destroy a drone seeking them out. This should be much less controversial than the first drone, since it's not operating on a human target. Now the first drone needs to be able to eliminate one -- and shortly, multiple -- drone(s) before then destroying the human target. The defending drone(s) will certainly increase in number and firepower. It's basically a straight line to automated warfare in the streets with no regard for collateral damage.
The hurdles, of course, would be regulation on the ability to purchase drones, and the permitted level of firepower. I don't like the idea of us negotiating the laws around that while drones are causing collateral damage in an urban (or any) environment.
This is literally about killing another human being and you went with “Well yeah but”.
As hn_throwaway_99 mentions below, the "Slaughterbots" video is not too far off from how STM presents their equipment: https://www.youtube.com/watch?v=9fa9lVwHHqg.
Had a similar reaction once before, after randomly walking in a military exhibition. While I understand the need for defence capabilities, and I can be impressed by the engineering, I don't get the triumphant glitter surrounding the showcase of killing machines.
You know, the other day I saw a news headline about one of the world's biggest arms dealers being arrested... and the first thought through my mind was, "What? They arrested the United States?"
Being a small fry, independent arms deal is grounds for imprisonment, but doing it as part of a nation-wide effort will get you wealth, medals, social status and government funding.
We still haven't learnt to live with one another - after all these millennia on earth.
https://www.youtube.com/watch?v=K0om03oXX_8
Awesome launching system.
Now, what can oppose those drones, and in particular by the probable opponents in the region? Manned air fighters, other drones, and jamming. In all those cases to preserve and maintain superiority the drones have to get fully or at least partially autonomous. Note that the Turkey isn't participating in the stealth race even though they could have gotten F-35 if they behaved, and the countries like Azerbaijan and Ukraine for example have no chances of getting stealth, so drones is the [only] way here, and autonomous drones is just the next step in the development of those weapons what their national security (or regional superpower status in case of Turkey) will be relying upon.
> The drones were operating in a “highly effective” autonomous mode that required no human controller
I think "fully autonomously" is a bit of a stretch for this particular story. I'm falling to see the difference between that and a heat tracking device or a laser guided missile except for the sophistication of the target seeking system. In this case a human designated a target and pulled the trigger, as opposed to an autonomous decision to attack humans.
But that it is clear that we are at only an inch of having drones creating no mans lands on demand.
I can't be the only one.
Please tell me I'm not.
They are now producing knockoff's of said drone.
Edit: 69 was a typo, but I'll keep it.
And yeah, as I wrote my comment I also envisioned self-recharging drones as a logical next step. If you're going fully autonomous, why keep any requirement to come back to base? The one in this story was kamikaze, but if you needed something with more of a punch that became expensive enough to require reusability, then you could have it come home, or just station autonomous ammunition stations at strategic locations.
These fall to the ground after 30 mins.
Unexploded bombs also bury themselves.
These will be attractive to children (boys) but since they can only go 5 km the children are already part of the battle.
Sure I'd add a further mode to land early and have a low power sensor to blow up when it detects a person in some cases but there's little reason to think this story is even true yet.
Unexploded ordinances predominantly kill young boys.
Even though it saves lives how we direct education campaigns, Twitter won't allow us to discuss this? What a world we live in, it's honestly falling apart.
"the effects of unexploded ordnance in Afghanistan. In this case, 81% of those injured were civilians, 92% were men and boys, and 46% were younger than 16" - https://reliefweb.int/report/world/differing-effects-explosi...
> child labor is limited.
For labour it's a farming and digging issue for adults but for the children killed it's mostly about play I'm not sure where this is coming from?
As I said given Kargu-2 are going to be on the surface or returned to base they will be quickly cleaned up. Also unless they have gone to suicide mode and missed they should be relatively harmless.
Without added programming they are not like mines.
I think the more dangerous version is drones doing the target selection (or actually any AI, on-board or not). And here it doesn't even matter much if there is a human in the loop, because he is likely to overestimate the drones confidence and press "confirm" most of the time. I think there were already reports of heuristic target selection (with a man in the loop).
Though I'm surprised we still need to kill people. I thought wars are won by technology nowadays, so what good does it do to kill a person? But maybe after all individual people are still important to organizations and it's a deterrence.
I’m more troubled by the way the military turns people into order following machines (an extreme exaggeration), than the fact that it had actual machines that follow orders.
As long as there’s a clear chain of command (responsibility and accountability), it’s not that different.
It will probably be like stock trading algorithms: a slightly more good than bad will be produced (more stable markets), but not a huge game changer.
Human operated drones have long been criticized for dehumanizing conflict which allows for escalating harm and harming unnecessary targets without issue.
Similarly war atrocities are somewhat limited to what you can convince your army to do. Not so with drones, they won't revolt and they won't tell the news either.
Perhaps it would be nice if conflict was just drones versus drones, but that is unlikely. War doesn't happen where you all arrive to battle in a field somewhere, it will be drones flooding towns and human combatants and civilians all in the mix.
However, I do believe the US military has a very strong “culture of accountability” - much more so than US police, US politicians, US civil servants, US scholars, or anything else in the US.
I suppose autonomous weapons might not share that culture, but whoever is deploying them does.
It’s worth pointing out that for a long time, drone strikes were the provenance of the CIA which lacks a formal chain of command.
As long as the decision to deploy an autonomous weapon, and what’s its parameters are, is set by an accountable agent, I don’t see a ton of difference between the uncertainty in an autonomous weapon, and the uncertainty present in the chaos of war.
See: US bombing of Chinese Embassy in Belgrade.
Honestly, an autonomous weapon might be much more like to avoid non-military targets (like the Chinese consulate). If so, great!
Edit. I reread your comment. Regarding atrocities, automous weapons can care about atrocities far more than non-autonomous weapons. A nuclear bomb or stick of dynamite doesn’t care who it’s blowing up. An intelligent targeting system does.
A properly trained AI, with a secure audit log, could serve as a strong deterrent against violating the laws of war.
This drone can probably be had for < $10,000 and it can be set it off from the boot of a car by a 10 year old with minimal training.
Price.
Obviously some scary as hell scenarios as well, but just pointing out there might be some benefits.
Any number of countries with animosity against each other now can attempt to neutralize their opponent without much effort.
Before there a deterrent was to invade and overcome you had to amass quite a bit of materiel, supply chain as well as have competent leadership.
now all you need is a plan and some drones and you can strategically weaken an opponent.
Countries could likely do the same thing today with explosives planted at key timings/places. And drones definitely open options in this but if a country was to act this way I think they would be just as likely with non-drones.
Overall I agree this is an extremely dangerous technology for humanity. Generally anything that will allow a small number of people control a larger number, history has show this will be often abused.
This may not remain true for long, if food and package delivery via drone become commonplace.
Many have feared 3d printed weapons, but it has been years since we have seen functional ones and nothing changed.
The interesting thing about this is that nobody pulls the trigger. You don't get killed because somebody wants you dead; you get killed because lots of people want you dead. And then everybody is hidden in the mob, and not really responsible for your death; after all, if 49 other people didn't also want you dead, you'd still be alive.
If studies on diffusion of responsibility (eg. Kitty Genovese) are true, then it's likely people will be pretty free with their nominations, knowing that the person won't actually end up dead unless lots of other people also nominate them. Mass carnage is likely to result.
https://en.m.wikipedia.org/wiki/Assassination_market
Guess what, eleven years later society has not collapsed and there are no assassination markets. Turns out that murderous psychopaths are not as commonplace as the statists would have us believe, and those that exist are not really into it for economic reasons.
It would make more sense to do this privately and anonymously, so Monero or funds obfuscated with Tornado.cash on the Ethereum network would suffice and a computer can still react to payment received and begin operating.
The first self-guided antiship missiles might also qualify, but they were (iirc) lock-on before launch weapons. (Operational in the Pacific theater during WW2, even!)
The original source looks to be https://www.newscientist.com/article/2278852-drones-may-have..., but that's hardwalled and so off topic here (https://news.ycombinator.com/newsfaq.html, https://news.ycombinator.com/item?id=10178989). If there's a better accessible source, we can change it again. I've used the NS headline since it's the most neutral.
So they will enable waging war in situations where it was previously impossible on humanitarian grounds.
Nice selling point.
Fuck that.
I honestly can't decide which is worse: Weapons so terrible nobody wants to go to war, or weapons so neat that war is the obvious choice.
Unfortunately, I'd have to put my money on munitions like these being used more often for targeted assassinations of <insert name of prominent person here>. It seems on the surface, very effective. And I could see where it would be hard, if not impossible, to trace. I think people in the future will think twice before running for mayor or agreeing to be a sheriff or police chief for instance.
For example, once the cartels build some of these, I'm not sure what kind of odds I would give anti-cartel leaders. Probably would be even worse in the US given our acrimonious dichotomy. I could see tit for tat assassinations going on all the time.
Of course, all that also means that a lot of work will likely be put into countermeasures. So who knows how all this will develop I guess?
Unfortunately I don't really think this matters much in war time.
The best realistic option here, in my opinion, is to require sensible constraints. E.g. hard-wiring war zones in them - a separate circuit could prevent firing if the drone is outside certain area. You could decrease the attack surface for such system by relying on a Kalman filter getting it's data from a dedicated accelerometer and a compass.
Of course most recently Syria is a horrible exception to this track record, and it remains to be seen if this will embolden others to do the same. Nonetheless, the prohibition does show that such agreements can at least reduce the usage of certain methods of waging war.
But I don't think hardware or software constraints be effective: Once one side of a conflict decides to break the a prohibition about using autonomous drones, I don't think they will care about hardware protocols. There's also the fact that warfare generally does not have static boundaries and no one willing to use this type of weapon would concede to their uselessness once the fighting moved elsewhere. Especially because their enemies would simply & quickly change their tactics to work outside that zone.
Chemical weapons are not used because they were ineffective even during WW1 and make even less sense now, when there is no trench warfare and targets are generally highly mobile. Conventions have little to do with this.
Certainly they were effective enough that Japan used them in thousands of attacks against Chinese forces. Effective enough that the US and Russia and probably others actively researched and stockpiled them for decades into the cold war. They were considered a significant threat by the US during the Gulf War: Iraq had a history of using them against Iran so the US took many precautionary steps to protect or inoculate their soldiers against them. Too many precautions as it turns out because those protection methods are themselves credited as a major factor in Gulf War Syndrome which impacts hundreds of thousands of US soldiers. The US certain believed in their efficacy 70 years after the trench warfare of WW One.
I don't know where you get the idea that they are not effective, or only effective under conditions if trench warfare. Actual use shows this is not the case.
Not completely, and for what is, in even human time-scales, not at all a long time. We'll see where it goes, but I'm not hopeful in the long run.
Regardless of what comes after this, we can say that the prohibition worked reasonably well for at least 100 years. I'd certainly be happy if we got even 25 more years before fully autonomous lethal drones became common place, and a prohibition on such weapons could buy us so e time on the issue.
I don't disagree with your point, but this part highlights how hard this is in practice. When does "the practice end"? When the war is one? When the current offensive ends? When we believe the devices aren't being produced anymore? This immediately reminds me of how Chamberlain treated Hitler.
You can also go after the people behind the orders to do these things and bring them up on war crimes. I find the former option much more ethically reliable, as the later frequently suffers from the fact that war crimes tribunals are generally not convened against the actions of whoever wins, while the winner is all to eager to punish their enemies even beyond defeat.
But none of these options are perfect. We live in a world where bad people can do awful things and never suffer the consequences. Justice isn't dead, but it has never been equally applied.
The video on the STM site is kind of lame (it shows a drone detonating a cluster munition after homing in on a high contrast target with humans spaced around it. That isn't to say that is the limit of its capabilities but it doesn't inspire confidence either.
All in all this has been a surprise to exactly nobody however. Given the cameras these things use are easily damaged by only moderately high powered lasers (like the ones used in laser light shows at rock concerts) I expect we'll see optical counter measures fairly quickly. Then there is the whole "emp gun" stuff that was being shown at the UAV conference in Las Vegas in 2018 to consider as well.
All in all a sad step toward more varied ways of killing each other. Sigh.