Hacker News Reader

Comment by tempnow987 | Hacker News Reader

If you are a major power with the capability to make 100K+ killer drones or similar per year, going to be hard to get you to sign this.

What is the drones annual salary? What is their annual housing cost, base allowance costs, VA medical benefits (lifetime)? How upset will your population be if one is "killed". Get the beancounters involved, and killer drones are going to be very tempting. For $75B per year or so (10% of US military budget) what kind of killer drones could I buy and how many?

The cost to deploy one F35 is astronomical. The training costs and lifetime costs for the pilots alone is heavy. The F-35 dedicates MASSIVE payload to life support for its human occupant. What kind of killer drones can you build with none of that mass, none of the gforce limits? None of the redundancy requirements? Do you need an aircraft carrier (thousands of sailors! -> needs destroyers and a battle group). No?

Can we design these to drop from space? Park 50K in space for quick de-orbit and insertion globally? Can you do this with people these days easily?

Going to be a scary world.

My quick guess is US, China, Russia, EU (maybe), India as places with enough capacity for this type of next gen battlefield? Maybe EU will move too slowly, so it'll be US / China / Russia?

View on HN

After WWI, the mass horror caused by certain types of weapons like chemical weapons were enough to get all powerful countries to agree we didn't want to see them on the planet.

Hopefully we'll have a chance to recover from the new horrors we are creating too.

Whether you have the capability to make them or not, either the US, Russia, both, or someone else will be HAPPY to sell you a bunch. Which is another reason those with the capability to make them want to keep it, to make the money, maybe even a more significant reason. And countries that _don't_ have the resources to pay for F-35s or air craft carrieres have possibly more to 'gain' here. Killer robots will be a great equalizer in the ability to create a nightmare hellscape, making it accessible on a much cheaper budget.

> Killer robots will be a great equalizer in the ability to create a nightmare hellscape

I imagine most people here have already seen the Slaughterbots video: https://www.youtube.com/watch?v=9CO6M2HsoIA

A more conventional type of attack would be using large numbers cheap cruise missiles. If you can make them for $20k each then $1bn would buy you 50,000 of them, which is probably a good deal more than the number of anti-aircraft missiles your adversary has, and certainly a lot more than their number of fighter aircraft. Therefore firing off a lot in one go would overload their defences and many would get through. If aimed at infrastructure target, they could cripple a country's economy, and how soon after that do people start starving?

I hadn't seen that video, thanks for pointing it out.

Eh, the rest of the world would probably gang up on you at that point, it'd be pretty suicidal.

I think a more likely scenario is killer robots are making the places that state and non-state actors already "get away" with turning into nightmares just that much more nightmarish, and that many more of them. Whether it's the USA in Afghanistan or what goes on in Sudan or Somalia. More and more places like that, which will be more and more nightmarish. Would the USA even have to "give up" on Afghanistan with that tech doing what the drones alone couldn't? But now also the smaller-budget actors like Somalian warlords get their go at it. It's gonna be great.

I believe "slaughterbots" may be able to better evade certain types of air defense (e.g. CIWS), in exchange for maybe being more susceptible to other types of air defense (EMP?).

Additionally, hypothetical slaughterbots would have much lower civilian casualties.

You just put up chicken wire and the slaughterbots get stuck. Most of these weapons have very low tech countermeasures. Where they're effective is when they're not expected. That might work once.

https://spectrum.ieee.org/why-you-shouldnt-fear-slaughterbot...

Maybe. The 'slaughterbots' of the future as depicted use explosives to blow holes in buildings. I'd imagine they could arrange multiple in a circle to blow an arbitrarily large hole in fencing or netting.

Also, usually chicken-wire fences just go around what's being protected, not necessarily over it (except for small farm animal enclosures, like actual chicken coops). It's hard for me to imagine entire buildings / neighborhoods / cities with chicken wire covering over the tops of the buildings in a way that couldn't be quickly breached with explosives.

Or drones could unroll detcord in a circle against the fence as well.

If somebody were to pull off a slaughter bot attack there would suddenly be a hundred other countermeasures beyond chicken wire. Slaughterbots as portrayed here are as overhyped as full self driving and the AGI.

>After WWI, the mass horror caused by certain types of weapons like chemical weapons were enough to get all powerful countries to agree we didn't want to see them on the planet.

They were banned because nobody liked them and they weren't tactically useful.

If they were useful they wouldn't have been "banned".

Let's not forget the UK, which is number two[1] for arms exports.

[1] https://en.wikipedia.org/wiki/UK_arms_export

I'm sure the US and Russia will include backdoors into all of the drones they sell which will changes these dynamics a bit. Probably more likely to have proxy wars then rogue wars.

The rest of the world, like Ethiopians and Ukrainians have been doing, would just buy from more... Understanding actors like Turkey instead.

> After WWI, the mass horror caused by certain types of weapons like chemical weapons were enough to get all powerful countries to agree we didn't want to see them on the planet.

Yeah. Kind of. Officially. In primary school books. Chemical weapons (and biological weapons) don't work¹, in addition to producing horrific images and stories.

It was easy to ban them for this combination of reasons.

Killer robots? We don't know yet, but they have the potential of working excellently well. (Although it depends on various reliabilities, the logistics chain required, activation time, percent deployable time, and failure modes. Probably among other things.)

If they work, it will be hard to get them banned. The measure of their effectiveness will be their ability to neutralise the enemy's military capability without destroying economic infrastructure (which you will want to take over and use for yourself. If you don't, there are nuclear weapons for that).

1. https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...

> My quick guess is US, China, Russia, EU (maybe), India as places with enough capacity for this type of next gen battlefield?

Turkey has already developed their own autonomous killer drones (which have in fact killed people, in Libya). So any country with their GDP and level of technology probably could too.

According to Wikipedia, Turkey has the 20th largest economy, and 72nd by per capita GDP. So a lot of countries could develop this technology if they wanted to.

A human can make smart decisions, even if his radio is jammed and he has no connection to base.

What does a "smart" killer robot do, without remote connection, in an unexpected situation? And wars are full of those.

And massive jamming (and hacking attempts as well as sensor confusion) is to be expected. So you either have armies that stop fighting regulary out of savety protocols (how likely is that in a war?). Or they do not stop fighting and make AI decisions on who to kill.

As far as I am aware, every side knows this and every side is very uncomfortable with this. But it seems every side feels pushed to advance.

And sure, humans make errors, too. But autonomous robots have a tendency to work most of the time - and then unexpectedly make a exponentionally mess.

That's why most robots who can do damage, work in very constraint settings.

But even harmless roombas did make a mess, by swiping dog poo all over the house, while "thinking" it is cleaning.

Swarms of killer bots out of control .. is a scenario no one wants. And military is about control, they do not like things they feel they cannot control. So there will be caution. Hopefully it is enough, to not trigger bigger, potentially nuclear mess.

> What does a "smart" killer robot do, without remote connection, in an unexpected situation?

Possibly kill anything that looks like an enemy, then attempt to return to base or self-destruct.

I expect killer robots will be programmed with geographic areas within which they are allowed to kill things.

"killer robots will be programmed with geographic areas within which they are allowed to kill things."

What about sensor jamming or confusing? How to make sure, it is in the right "kill" area?

And if you designate a zone as free to kill everything, you might as well nuke the area.

> What about sensor jamming or confusing?

This will no doubt happen.

> How to make sure, it is in the right "kill" area?

If a robot knows where it started from it can use dead reckoning to know roughly where it is. It can also use landmarks it identifies, or celestial navigation.

All of these should give it a reasonably accurate idea of its whereabouts.

> And if you designate a zone as free to kill everything, you might as well nuke the area.

No, because a lot of UGVs will be used as area denial weapons, i.e. as a smart minefield. They might typically stay where they are and monitor than no-one uses the area. Nukes can't do that.

"as a smart minefield."

Smart as in kills anything that moves, sure.

"All of these should give it a reasonably accurate idea of its whereabouts."

But if you add high speeds and bad weather to the scenario, together with highly populated areas, reasonable accurate becomes a relative thing.

Unless the drones as a consequence mostly fight themself in remote areas.

Might be better than humans fighting in urban areas, but reality most likely will be robots vs. bad equipped humans in urban areas.

I agree with your analysis that this is a core issue, but something that bothers me with these discussions is:

> If you are a major power with the capability to make 100K+ killer drones

"The major power" is the people that make up the country, which largely don't support this stuff, but there's a handful of decision-makers that can push for this even when the people don't want it. I feel like it's helpful to focus on how it is individual people making these decisions that are going to harm others - not just a result of a country having a certain amount of power/capability. Any one of those people could go public and say that they're not going to let their country go down that path and that they'll discourage others from doing it, but they choose not to for fear of the results.

> "The major power" is the people that make up the country, which largely don't support this stuff

The populations of most countries probably don't want to attack foreign countries, but they absolutely do want to avoid being attacked by foreign countries, and if having scary weapons will do this, most will be for them.

The last thing the defence industry wants is for governments to spend less on warfare. If these robots ever get deployed to a battlefield it'll be as something like mechanised landmines or assistive tech for warfighters, not a replacement for existing tech.

The total costs will not go down, but the effectiveness increases substantially until everyone "has to have" the same expensive technology and funding cannot possibly be diminished lest our country becomes vulnerable.

The dreaded "arms race" doing its thing since the dawn of humanity.

The issue is not technology, it is that some states have objectives that're irreconcilable with other states. For example, Russia and China both want to conquer other countries (e.g. Ukraine, Taiwan), something that many countries are against (obviously including the ones threatened with being conquered).

I'd argue against virtually all the points you make BUT you actually miss the main point of this:

this is about making autonomous decisions in the field beyond just local navigation and such. Your arguments hold for non-autonomous drones and autonomous ones to pretty much equal degree. But you don't make any point about why humans shouldn't remain in control of combat decisions.

Consider US vs PRC scenario - naval battle over ocean where theatre area is cordoned off as no-go zone for commercial traffic. UAVs massively out maneuver human pilots. Massive electronic warfare disrupts data links and force remote human out of loop. So UAV falls back to onboard sensors to operate autonomously to finish mission to sink enemy ships. Well preferrable enemy ships that match onboard profiles, but might as well as sink all vessels within a no-go region. No real need for human decision making if goal is already well defined and the environment permissible. Vast percentage of the earth (bodies of water, minimally occupied lands) can easily be turned into no-mans land for UAVs to battle with little risk for collateral damage. It's the same permissible environment for tactical nukes, which is still part of doctrine / planning for anyone that has them.

Right. Gotta realize we humans have to give these UAVs the space they rightfully claim on this planet... /s

Because in war humans already move themselves out of those decisions.

Landmines and huge minefields. This is death with no human involvement. If a commander draws a box on a map and says deny the ability of the enemy to cross this box on our flank, does he care particularly whether it's mines (that are a pain to deal with after things quiet down) or if it's killer autonomous drones?

Plant security?

Border patrol?

Think of a rich guy with a private island. Security is wildly expensive, but they are paranoid (check out the security some of the Russian oligarchs run - it's pretty much a mini mercenary force). Would they deploy some automated AI that kills anyone unauthorized (after some warnings?) My guess is yes.

Except they aren’t cheap at all (even if they should be) and the defence contractors building them aren’t exactly doing an awesome job for all that money.

Remember this? https://www.wsj.com/articles/SB126102247889095011

> If you are a major power with the capability to make 100K+ killer drones or similar per year, going to be hard to get you to sign this.

Surely the individual decision-makers are, as with all human beings, averse to a world swarming with killer robots.

That states make indefensible decisions with much greater frequency than their individual statesfolk is so obvious at this point; it's time to simply stop tolerating their existence and move more swiftly into the next stage of information humanity.

I'm a citizen of one of the states you mention and I can assure you: amidst all the disagreement and political polarization, there is complete consensus on not wanting a world dominated by killer robots.

> there is complete consensus on not wanting a world dominated by killer robots.

That and nukes. It is ok to not want these things, but to survive as a nation state, these things will become necessary developments in order to survive. If GP is correct about some of the capabilities we're likely to see, then any nation state without these will need to begin development in order to ensure survival. Is that an arms race waiting to happen? Yes. Is there a better way? No. The alternative to an escalating arms race is actual war.

We will never be done with power hungry ideologues, we will never be done with Communists whose sole mission is dominion over the whole world, we will never be done with petty despots (looking at you, Putin) who so wish they could play a meaningful role on the world stage. All of these have every reason to develop a new super weapon to tilt the balance of power in their favor.

Iran, for example, has started its centrifuges again. They also have some new ones that spin at 10x the speed of their old ones. They are very close to possessing a nuclear bomb. Up until the end of the Trump presidency they had halted production of uranium and talks were going well. Biden has about the weakest foreign policy of any president in the last half century.

Part of that effective foreign policy is the military might the US has. (Why do we have 5 carrier battle groups? Why does Britain only have one? Why does Russia have a single carrier that's in rather poor repair?)

If killer drones are an effective war-fighting tool (an it appears they are), they are inevitable. What we need to ensure is that they are used as humanely as possible.

> That and nukes.

Yeah, good point. But I can absolutely envision - without much difficulty - a megatons-to-megawatts motion swiftly bringing an end to this period in Earth's history.

And more generally: maintenance of arsenals of weapons of mass destruction is _very_ costly. Economics of energy production and storage are perhaps a viable mechanism to add pressure in favor of disarmament.

> (Why do we have 5 carrier battle groups? Why does Britain only have one? Why does Russia have a single carrier that's in rather poor repair?)

We can also summarize questions of stagnant geopolitical asymmetry this way:

How long will the internet abide states as nodes of economic and social standing?

Trying to shoehorn a previous world order into an information age society seems like a colossal waste of time in service of nostalgia.

> nostalgia.

Nostalgia does not guarantee me personal autonomy. Neither does it guarantee me a right to self defense. Nor does it guarantee a free practice of religion, free press, or a right of association.

Ensuring the survival of a nation state that adheres to the US Constitution will guarantee those things.

> How long will the internet abide states as nodes of economic and social standing?

Take a look at China and their authoritarian take on "the internet". "The internet", that is the masses that make it up, do what they're told, consume what they are allowed to consume, demonize those they are told they should demonize. Goebbels would be jealous of the machine the modern world has developed.

> Nostalgia does not guarantee me personal autonomy. Neither does it guarantee me a right to self defense. Nor does it guarantee a free practice of religion, free press, or a right of association.

> Ensuring the survival of a nation state that adheres to the US Constitution will guarantee those things.

This is _precisely_ the uncultured nostalgia I'm suggesting is unworthy of our defenses.

I absolutely support your right to bear arms in your defense. To be free in your religious exercise and associations and speech and press, and in the security of your effects against intrusion.

But the US is not, as you say, a nation state that adheres to the US Constitution in any broad and universal sense.

We have two million people living in cages. Five million more living under supervision of a system of slavery rebranded as carceral rehabilitation. Drugs and medicines and sacraments subject to capricious regulation at best, and often outright prohibition. Fiat currency. Arms which are practical as tools of self-defense unavailable to a quarter of the population.

And I'm not sure that anybody other than (some) white men have _ever_ had these things securely in their civic possession.

So the question is: _can_ a state be the bearer of insurance for these things? Especially now? Why can't we protect each other? Engage in commerce regulated (which is to say, made regular) by the internet alone? I have no idea what the benefits are of the continued existence of the US State, but the downsides are absolutely evident and very real on a daily basis.

> Take a look at China and their authoritarian take on "the internet".

I take your point, absolutely. But doesn't China and the Great Firewall also provide us with an example of failure of authoritarian tendencies as they apply to the internet?

May 35th lives on largely as an internet meme. The official truth may dominate in circles where it can be violently maintained, but those shrink in the face of connectivity, no?

You've hit the nail squarely in your comment. The US is hideously broken. Notice I said "a nation state that adheres to the US constitution". I am not unaware of the failures of the United States, its abuses of power, and the hideous humanity expressed by its elite class (pedophilia, child sacrifice, etc.).

> _can_ a state be the bearer of insurance for these things?

Yes, but it requires the governors of the state to be moral. The peoples of the US have completely failed in passing moral ideologies to their children and grand-children. This moral failure has led to every disintegration of orderly running in the bureaucracy of the US and its states. If the pieces (the people) are immoral, so is the machine.

> And I'm not sure that anybody other than (some) white men have _ever_ had these things securely in their civic possession.

We pretend the opportunity is still to be had. But taxes and inflation steal most of the commoner's earnings. Taxes and inflation see to it that wealth (not necessarily dollars) flows to the elite class. Regulations ensure no one can fight the loss of wealth.

I think people may sometimes forget thag, so far as 95% of the population goes, the internet is still a communication and commerce channels, with endpoints in meatspace. some endpoints are in liberal democracy, some in totalitarian regime. to overthrow the latter you need revolution or nationstate interference.

Policy change-politik-is a function of political will and political efficacy.

the internet is just an information conduit, end of day

> I can absolutely envision - without much difficulty - a megatons-to-megawatts motion swiftly bringing an end to this period in Earth's history.

I can't figure out if you're talking about a nuclear war changing the world, or a nuclear disarmament process that somehow also leads to energy production.

I found the rest of jMyles' comment equally baffling. My advise to people is to talk as simply and clearly as you can.

> I can't figure out if you're talking about a nuclear war changing the world, or a nuclear disarmament process that somehow also leads to energy production.

My apologies - the latter.

I think you're forgetting that the only thing the internet does is make communication easy. The material realities of where we live, our cultures and belief systems determine how we split into groups like countries.

So while the internet certainly enables us to be more connected as a species, it doesn't eliminate what makes having nations a valuable construct for human societies. As long as our physical environment matters, nations or similar constructs will be around.

> It is ok to not want these things, but to survive as a nation state, these things will become necessary developments in order to survive.

Yes, exactly.

> If killer drones are an effective war-fighting tool (an it appears they are), they are inevitable.

I have come to the same conclusion.

I see the Cold War never ended in the US. The communists you're decrying engaged 80% of the German army and were responsible for 7/10s of Nazi deaths on that front.

It wasn't the USSR that dropped two nuclear bombs (one uranium, one plutonium) on Japanese civilian targets when the Japanese were willing to surrender on the sole condition that they keep their emperor (a religious figure)

In the name of "stopping the spread of communism" the US overthrew popularly-elected governments, sent death squads into Latin America, and killed millions of civilians besides the ones they permanently maimed. This is besides the typical "make their economy scream" that kills civilians as well, in a way harder to quantify. Cuba is still the victim of this decades after the failed Bay of Pigs.

We have technically been in a state of emergency since the 50s with the undeclared Korean War, and for what? The permanent wartime economy hasn't benefited anyone besides the military-industrial complex. American imperial designs in Vietnam, Laos, Cambodia, Cuba, Yemen, Syria, etc. were all enabled by this escalation mentality and the crimes against humanity that were committed there are a permanent stain on humanity

> the Japanese were willing to surrender on the sole condition that they keep their emperor

Heh, that's quite the narrative.

https://en.wikipedia.org/wiki/Ky%C5%ABj%C5%8D_incident

What action can actually be taken towards that goal? Your own country can avoid creating these robots. But you can't know that any other country isn't pursuing them. Even if you could, the political capacity to sanction them for it is quite limited. So the most likely outcome of trying to ban them is that every country assumes every other country is still building them in secret, and thus every country still builds them in secret. Eventually, the fact that they are building them in secret becomes public, and ultimately nothing is done about it. Having a world with killer robots in it is bad, but a world where your enemies have killer robots and you don't is far worse. The decision makers will all be very cognizant of that fact.

> But you can't know that any other country isn't pursuing them.

Indeed, what would happen is that if such a treaty was signed, some other countries would develop them anyway, as hidden modes for the use of their otherwise non-autonmous UAVs and UGVs. So any such treaty would essentially be handing the world to countries that cheat -- which would likely include Russia and China.

This kind of reasoning is circular. It justifies any and all escalation on the chance that others are doing the same. Also baked into the assumption is that fire fights fire.

I don't want my state to produce killer robots. Even if other states do. I can only impact my own. It's no secret that the US disproportionately engages in this "warfare" (read: asymmetrical slaughter) and even under a "liberal" head of state (Obama) civilian casualties of our drone program was 90%.

You're also assuming regulating killer robots means a ban, when the international community could restrict deployments or justifiable uses. It's much more realistic even if rogue drones are created.

The decision makers are duly elected representatives of their constituent body and it is beyond time they acted like it. Otherwise we're going to keep having more Vietnams, more Afghanistans, more pointless civilian death.

It's possible to be more cynical about this, but yet to come to a more optimistic conclusion.

If the world's great military powers publicly swear off "killer drones" (let's assume we can meaningfully define such a thing), then they will continue to develop them in secret, but they won't use them in every little border skirmish: every Iraq, Syria, Afghanistan, or Crimea. They'll keep them for the big game.

This has two good effects (assuming further development of killer drones is bad). Firstly, those of us unfortunate enough to be caught up in the horrendous bloody conflicts I callously described as "border skirmishes" won't additionally suffer from killer drone attacks. Secondly, development of the drones and the military doctrine surrounding them will be stymied: you can do a lot in the lab but any technology develops so much faster with the feedback loop that comes from using it in anger.

Moreover, if there's a big war we're all screwed anyway. If that's your focus, you need to argue that drone availablity makes a big war more likely I'm not convinced that it does - making war cheap makes small war more likely, but all the decision makers know the cost of a big war.

Edit: compare to anti-personnel landmines, or chemical weapons. Both of those see less use because of weapons bans. You might say no, they see less use because the great military powers no longer need them - but you're missing a step. The great powers no longer need them, they decry them as inhumane, they sign bans, and then (almost) everyone stops using them.

> The great powers no longer need them, they decry them as inhumane, they sign bans, and then (almost) everyone stops using them.

See for example Bret Devereux: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...

Kill-bots change the moral calculus too. As an example, it's one thing to rush a cop on the assumption that human cop won't just gun you down. It's another to rush a programmed kill-bot. The latter case is obviously just suicide, which is morally on the person committing it, not the tool they use to do it.

I'm well aware there are many different ways besides the above one could argue it, but my key point is that automatons change the moral calculus, no matter what other premises you accept.

I think we need an outright ban, but I also understand how workable this strategy is.

Drones have the power to make WWI look like a football match. They could already be in larger use than people realize, it isn't like one would hear about it. There would be nothing to report.

> Surely the individual decision-makers are, as with all human beings, averse to a world swarming with killer robots.

The history of the use of land and sea mines in warfare suggests to me that this is not a universal we can assume.

>I'm a citizen of one of the states you mention and I can assure you: amidst all the disagreement and political polarization, there is complete consensus on not wanting a world dominated by killer robots.

If what people "want" is a nice sounding sentence slapped on a roiling pile of inconsistent sub-wants, then good things aren't going to happen.

Sure, nobody wants "a world dominated by killer robots".

Nobody wants "90% civilian casualties" from those killer robots either.

But if people are asked "would you rather Bob down the street get killed fighting in X or let a drone take the risks", what are they going to say? Yes, screw Bob? And that's assuming that the person asked isn't going to be drafted because there are enough volunteers.

If people are asked, "would you rather use drones and kill civilians, or use other methods and kill many more civilians", what are they going to say? Yes, more is better?

Reply on news.ycombinator.com