Hacker News Reader

Comment by colechristensen | Hacker News Reader

It’s the fundamentally different kind of danger from machines. They make mistakes we can’t understand.

I can understand a person being distracted, reacting slowly, not seeing me, etc. I can also mitigate this risk. Crossing the street I can make eye contact with a person and be really confident that they aren’t going to run me over.

Walking in front of a robot? I have no internal model for the kind of mistakes they make. It seems equally likely in any circumstance that they’ll make the mistake of running me down, and there is nothing i can do to avoid it.

View on HN

You do develop an internal model for the behaviour of machines, though.

Examples:

Elevators where the doors take extremely long to close on their own.

Elevators where the doors close too soon after opening.

Elevators where the limb sensors are not very responsive.

Cars that take a couple of tries to start in cold weather.

Automatic doors that are a little bit slow to open.

Etc.

Powered doors are one dimensional machines with perhaps one binary sensor and a motor driver, a person could implement a state of the set automatic door in an afternoon.

You would also have to be rather clever in order to get an automatic door to injure you.

A car is also a rather easily understood dynamic system, but that isn’t an issue either. We are all quite frequently in the position of being at between a few seconds and one heartbeat away from a car killing us... walking down a sidewalk, crossing the street, driving on the highway...

We understand pretty well the dynamics of a human driver. Even the drivers that make terrible mistakes we understand something of hope and why.

A robot driver though... the terrible mistakes it makes are mysterious and terrifying. They make the kinds of mistakes a human never would (and few mistakes a human would make).

but those dont happen at 60+ miles per hour

Okay, let's use another example: lifts don't move until the doors are closed. If the lift thinks the doors are open, it won't move. If it did, it would be an out of control situation just as quickly as a car misbehaving at 60 mph is. The expectation is that autonomous cars must be cautious.

my point is when we develop expectations and mental models of how machines work, we can use those to adapt and feel safe and in control. but that's less possible when the machine is typically moving at 60 mph and the mistake can happen without warning

Like elevators where the brakes fail. If it's infrequent enough, we tend not to think about it even if it does still happen from time to time.

I see your point but for me at least, my gut feeling is the autonomous cars are just scarier because they're moving at 60 miles per hour. Maybe it's irrational but this whole discussion isn't about rationality its about gut feeling

Yes! You can control for human risks in the tail scenarios. You can't control for the tail scenarios in AI because you don't know what on earth they are, they are a nasty surprise, thrown up by a model which, in the tail, we as engineers don't fully understand

Reply on news.ycombinator.com