Hacker News Reader

> * Robots do not act cruelly unless specifically instructed.

Well, sure, but have you used computers recently? It's often a fucking awful experience and while the computer clearly isn't trying to make your life frustrating that's often the result.

    PC LOAD LETTER

https://www.youtube.com/watch?v=5QQdNbvSGok

https://www.youtube.com/watch?v=N9wsjroVlu8

Do we really want the people who make printers to make autonomous killing machines?

Then again, the printer is just not working. It's not randomly printing things you don't want. In the same sense, a broken autonomous tank would just stop dead in its tracks instead of suddenly killing everything that moves. Those failure modes have always been very rare.

http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...?

> In this article I present in which way scanners / copiers of the Xerox WorkCentre Line randomly alter written numbers in pages that are scanned. This is not an OCR problem (as we switched off OCR on purpose), it is a lot worse – patches of the pixel data are randomly replaced in a very subtle and dangerous way: The scanned images look correct at first glance, even though numbers may actually be incorrect. Without a fuss, this may cause scenarios like:

... wedding parties being hit by drone strike?

But they shot AKs in the air as celebration! Only a mistake that kills an entire family tree.

Because any technological advance is going to be countered. Then at some point, we have drones killing drones.

Currently military numbers are limited by manpower. And military units (like let's tanks) are to some degree limited by money. You don't want to pour too much money on single tank, because you then put all your eggs on few baskets. And you also don't want to put significant part of your troops in badly armored tanks with food firepower, because then you will run out of tankers in no time. Currently the "good balance" is 10 000 M1A2 Abrams that cost 6,2 million dollars a piece. Those would need 40 000 tankers, which is tolerable amount of crew. Training them is costing and going to cost lots of money. Combined price of the thing is 62 billion dollars or tax payer money, distributed to several decades. Increasing the price of tank to 10 million would be stupid, as it's still vulnerable to certain dangers, but you now lose 4 million more in single blast.

Now let's imagine full autonomous tank fleet of M1A9 Abrams somewhere around year 2050. You are no longer limited by tankers. You are only limited by price. But as you lack crew, you need less armor. Lets say we have unit price of 4 million dollars. Now 10 000 units is only 40 billion and zero tankers.

Then Russia makes autonomous tank "T17 Kursk" and produces 15 000 of them to sell to China. (Or something else, I'm not fiction writer.) They are bit crap, but as trained tankmen are not required, they can easily produce higher numbers to compete with U.S.

But U.S. is not happy with second place. So U.S. gets another 10 000 Abrams.

But China is not giving up, they order another 15 000 Kursk.

And on and on. Until tax percentages of both China and U.S. reach 70% and all of it is poured to "defense". China, India, Brazil and Russia are all likely to beat U.S. in economic growth rate in the future, because it's easier if you start from the bottom. And they are all geopolitically ambitious. We are headed towards multipolar world and that means arms races again.

TL;DR: Autonomous killing robots remove limitations of arms race. This is "destabilizing" and draining.

People are not the limiting factor on number of tanks. See: the rest of the military. This is an implausible scenario.

It was simplification. But yes, the real shit hits the fan when infantry can be replaced with robotics. See "combined arms approach".

You do know that MBT's require a lot of day to day maitainace by the crew to keep them going - that's one reason why they still have 4 man crews.

Yes I do. It's oversimplified scenario to drive home the principle.

Money and perceived threat are big factors in the equation. But anyhow there is currently a limit on how much money you can throw at a Field army until you hit deeply diminishing returns. And numbers of enlisted limit how many field armies you can field. Robotics can push this limitation way further.

This is a short-term issue, which is widely known; autonomous self-healing support and supportable units are a coming.

Hah I suppose you think we have self healing cars.

Appears that you don't understand how auto companies make money; hint, it's not by selling cars.

Please enlighten me with more information regarding this scam that even Tesla hasnt expunged

Auto industry makes most of it's money from enabling auto ownership, not selling the autos themselves. For example: http://www.forbes.com/sites/jimhenry/2012/02/29/the-surprisi...

Believe what you want, but comparing auto industry to the military systems is meaningless in my opinion.

Thank you I was not aware of this.

right... so your going to have full human level ai robots that can replace humans exactly when ?

Hardware is limited by resources, which are neither limitless or instantaneous. That said, I agree the cost of war will rapidly increase, as will the desire to control resources; China already sees this, though they're not good at fighting economic wars.

I'd argue that China, Russia and the US have already passed this point w.r.t. nucelar weapons: Every side is sufficiently armed so that a war between any of them cannot be won. As soon as you start winning in a conventional scenario the other side might as well use nuclear weapons, just to ensure you'll lose too. In short a war between nuclear powers makes no sense, since the "victory condition" has been removed.

I'd assume that this is the same in asymmetric conflicts: If you are fighting a guerilla war, you "win" when you kill some of your enemies troops and demoralize them. However if enemy sends only machines, then cannot win as long as the enemy is capable of producing more of these autonomous weapons than you can destroy. As you've said the cost savings do enable that. Hence the guerilla side also loses its victory condition. This is a scenario I'd actually like to see, since it might put an effective end to the kind of fighting we've seen in the middle east.

Your argument is of course correct when we consider two countries which are both capable of producing such weapons and neither side has a nuclear weapon. Saudi-Arabia and Iran are candidates which could probably fall into this kind of an arms race and that would indeed make the situation there even worse.

"Hence the guerilla side also loses its victory condition. This is a scenario I'd actually like to see, since it might put an effective end to the kind of fighting we've seen in the middle east."

As a Finn, this is exactly the kind of stuff I don't want to see. Russia can win conventional war against Finland any day, if Finland would be absolutely certain to lose also the ensuing guerrilla war, we would lose our independence without fighting. Maybe not de jure independence, but practically yes.

Nukes are unusable as you said. This has not prevented conventional arms race and it's usage for proxy wars during cold war. It's unlikely it would do so in future. It might even make the whole situation worse, as now you can supply the "mujahideen" with so much more potent weapons. And you need less popular support to maintain similar levels of guerrilla activity.

Damn I hope I'm wrong on that. The possible counter could be autonomous aerial denial weapons. If they get better than autonomous assault weapons, we might actually see more peaceful world in the fashion of Kant(1). It might mean that Taleban might rule Afganistan again. But it would also mean that Russia can't wing Georgia, Egypt can't win Israel and Iran can't win Iraq. Westphalian utopia. :D

[1] https://www.mtholyoke.edu/acad/intrel/kant/kant1.htm It's really awkwardly worded essay, but the point is that if everybody defends and nobody attacks, we can get resilient world peace. With the downside of occasional minor conflict, but also with no "world government". If you start to think the election of president of world government, the whole idea seems pretty obviously oppressive and potentially very totalitarian.

Just a worthwhile note: guerilla warfare is not, by far, the only option. There are all sorts of techniques available to overthrow dictators and other occupiers.

A good example of this in real life is what Denmark ended up doing when occupied by the Nazis - non-compliance, plain refusal to cooperate with the Final Solution, and eventually even convincing the Nazis who were based in Denmark to sabotage their own country's plans.

And I also recommend the book "From Dictatorship to Democracy" by Gene Sharp which outlines many principles and techniques for non-violent resistance.

> we might actually see more peaceful world in the fashion of Kant(1).

I think we would just have nutters bombing us from the skies.

I should point out that if you are a perfectly immortal government that takes a strictly instrumental view of your population then the value of the typical 1st world citizen's life is going to be about $5 million. The government can't extract all that value from its soldiers but it should still consider their long term productivity forgone by them dying as much the cost of training them if you're being smart.

Of course that only enhances your point.

Arguably most conflicts are caused by people with much lower level concerns. Two opposing governments launching a drone war between them, due to some civilians being disgruntled at some other civilians, while these angry people watch from the sidelines, sounds very theatrical compared with governments lacking the political will or interest to stop some rebels from grabbing pointy sticks and fighting each other. Sort of how we have lots of countries with airforces, but current wars don't involve air vs air combat. People smart enough to use the technology don't seem to have that much taste for fighting other smart people.

Two opposing countries launching ICBM war between them never happened either. But that didn't stop people pouring incredible amounts of money into it. Until certain saturation point was achieved. Military spending is not much determined by war-fighting, but capabilities and threats.

Robots do not act cruelly unless specifically instructed.

I agree that in theory, killbots fighting killbots could lead to very clean wars. The problem is that the least powerful party will quickly resort to 'kill and torture as many civilians as possible' as a means of revenge, psychological warfare, terrorism, etc.

The ethics of robots will be the ethics of humans. More powerful and effective killing machines will amplify the behavior of immoral humans.

Do you expect the Pentagon be the least powerful party? Do you expect other partiesto stay away from aautonomous military robots if the US does?

Just to point the elephant in the room and place the correct problem there for discussion:

* Autonomous weapons subvert the basis of Democracy, making autocracies sustainable.

* Bugs on autonomous weapons can make them much more dangerous, to both sides, even on peace time.

Starting to think HN is full of sock puppets. Here's a downside, robots don't care about killing the population that paid to create them. Basically, every single claim you're making is false or is easily spun to be a negative.

The belief that people who don't share your opinions must be under control is a gross dehumanization, and not a terribly helpful way to begin a contribution.

Understand, fair enough.

It's remarkable that you had to preface your argument with "Just playing the devil's advocate here". It appears that the fear of AI taking life and death decisions is so great that merely making an argument in favor of it is seen as immoral. Doubly fascinating is the fact that this attitude has taken roots on HN of all places.

The more deterrents to war we have the better. Lowering the perceived risks of it seems dangerous and might in fact make it more likely to wage war. You need skin in the game to make you think twice about waging it. I worry the actual human costs could be much higher by removing humans entirely from one side of the fight.

Your logic implies, that the other war participants play by the same standards. I doubt, that there will be a standardization or approval process for the operating software across all armies, in order to ensure that the robots behave with some "moral limitations".

I think the better question is how do we stop war?

I think too often the casualties of war get dismissed as collateral damage. I'm for requiring politicians that vote for war to send their offspring into battle. Put some skin in the game so as to speak...