Hacker News Reader

Comment by wolverine876 | Hacker News Reader

> Advanced militaries have clearly indicated their interest in such systems. Reasons range from speed of response and enhanced situational awareness to the ability to overwhelm the defence systems of adversaries.

My understanding of experts I've read over years:

Western militaries, at least, have for over a decade worked on keeping humans 'in the loop', for the obvious moral reasons and also for effectiveness - human-machine teaming is considered much more effective than either humans or machines alone.

Yet they are developing autonomous machines because they don't believe they can rely on communications. The measure-countermeasure contest between communication and jamming leans too much toward the latter and it becomes a serious vulnerability: The machine army could be entirely shut down by simply jamming communications.

View on HN

Imo there’s a lot of rationalization. I worked on an autonomous quad copter learning to fly in a simulated environment back in 2013-2014. It would then target and track a target following it.

We then moved the AI from a simulation to reality and it worked very well. I was an undergraduate at university at the time.

Based on the training data we had (what looked a lot like predictor drone video), I imagine the tech is fully capable and ready to go as needed.

Everyone knew what we were building, but rationalized it as “oh this will be used for speeding tickets or tracking drug deals”… like sure, but I can see the AKs in the video lol

> speeding tickets

> drug deals

I can't help but think of how the world changes.

Think of speeding tickets, until the 70's, vast portions of the US did not have a speed limit. This changed during the middle east oil embargo with the introduction of the national 55mph speed limit (under the guise of saving energy). When oil supply stabilized and domestic and foreign oil became plentiful, it was not repealed. Now speeding tickets are common. A huge majority of people have gotten them.

I wonder if the tolerance value for going after a speeding car might become smaller as self-driving cards minimize offenses and ticket revenue drops.

In the same way, "drug deals" have led to civil asset forfeiture, causing lots of out-of-town travelers to give up their property, to the benefit of police departments.

so... today's "speeding tickets" and "drug deals" might become tomorrow's problem for a wider cross-section of the population.

>Think of speeding tickets, until the 70's, vast portions of the US did not have a speed limit. This changed during the middle east oil embargo with the introduction of the national 55mph speed limit (under the guise of saving energy).

They did not have a federal speed limit. State or local speed limits were imposed everywhere. Although pre-1973 limits were often a lot higher than the federal limit.

>When oil supply stabilized and domestic and foreign oil became plentiful, it was not repealed. Now speeding tickets are common. A huge majority of people have gotten them.

The 1973 law was repealed in 1995. And the mandate of setting speed-limits were returned to state-level

Nevada and Montana didn't have posted speed limits on many or any highways, respectively, until 1999.

Out of my knowledge area, but according to ChatGPT;

>[Well..], both states did have speed limits, but they had sections of rural highways with "reasonable and prudent" speed limits, which were often unposted.

>But they were not without limitations. Drivers were still expected to drive at a safe and appropriate speed for the conditions, and law enforcement could *enforce speed limits based on what they deemed reasonable and prudent.*

So to me than seems even more open to abuse than the current situation.

> I wonder if the tolerance value for going after a speeding car might become smaller as self-driving cards minimize offenses and ticket revenue drops.

In a future world where every car is self-driving and obeys the posted speed limit, I can see states and municipalities resorting to trickery and “hacks” like declaring a speed limit that changes randomly and unpredictably, so self-driving cars have trouble adapting and can get cited. It’s not like they are just going to allow that revenue stream to disappear.

> It would then target and track a target following it

Isn’t that just the standard “follow me” feature?

>> We then moved the AI from a simulation to reality and it worked very well. I was an undergraduate at university at the time.

There's tons of work like that (no offense meant) and it never amounts to anything much. If you used Deep RL or imitation learning, or really any of the Big Data, statistical machine learning techniques that are well known, then it wouldn't have worked in practice. Those approaches are extremely unreliable and fail to generalise to new environments robustly enough to be of use in the real world.

The reason why the article above can still ask the question "Have we seen active deployment of killer robots?" is that the question doesn't have a clear answer. If what you did as an undergraduate could really work as well as some people would like, there would be no question left to anyone's mind.

Again, that's not to bash your work. NVIDIA recently announced a new "moonshot" to develop humanoid robots for domestic use. That's because there's no such thing yet. Just a quick glance at the robotics and AI literature should be enough to convince anyone that true artificial autonomy is not yet possible.

Eh I know what we worked on ended up being deployed in production systems.

most of the team went on (not me) to found a company too: https://reconstructinc.com/

That a system is deployed doesn't mean it does what it is said to do. Specifically if we're talking about self-guided missiles or loitering munitions, those are designed to be destroyed on impact and so the requirements for autonomy are much smaller.

If you're talking about a gun drone that autonomously hunts down people and kills them, then that's not a technological capability currently available, at least not if you're interested in accuracy. Although, I guess that's the point, accuracy is often not a priority. Like early gun powder weapons part of the value of autonomous weapons is their potential for terror.

Autonomous weapons systems are nothing new. The US Navy put the fully autonomous Mark 60 CAPTOR mine into service in 1979. It can be placed in a fixed location and wait months for a target based on a passive sonar signature, then launch a homing torpedo to attack. Zero humans in the loop.

I agree that the line between that and automomous weapons is hard to draw. A simpler example is any precision targeted missile or munition. At some point every munition leaves human control, including artillery and bullets.

But is my 1979 calculator artificially intelligent because it can duplicate some cognitive things that I can do? There's a major qualitative difference where we find all the significance in the meaning of AI.

Still, some of the problems with mines, which are illegal or highly discouraged in many situations, are similar to autonomous robotic weapon systems: A fundamental law of war is discrimination between military and civilian targets. Mines are literally indiscriminate, by that legal definition; the worry is that autonomous robotic weapons may be the same. Arguably, they could be more discriminate.

I think one difference is in the 'robotic' aspect of autonomous robotic weapon systems (my terminology): Robots don't need to be mobile, of course, but those with significant mobility seem far riskier and deadlier than mines or artillery. Imagine one that, even from some malfunction, ends up in a city or a refugee camp.

As I recall, this type of mine (or a similar one) could target specific types of ships, or even a specific individual ship if enough data was available.

Reply on news.ycombinator.com