Comment by simonh | Hacker News Reader

Comment by simonh | Hacker News Reader

Parent Full thread

Humans can make risk/reward calculations that computers can't. Top take an extreme example, a given set of snow and ice conditions might be bad enough that any reasonable person would not want to drive in them. In such a situation, any well designed self-driving car would refuse to drive. BUT my wife is pregnant, it's a difficult pregnancy with a high risk of death for her and the infant, and she's suddenly gone into labour weeks early in a freak snow storm.

A contrived example, yes, but the point is there may well be conditions in which, given a wider context, a human would do things a computer can't because it doesn't have the wider context. The thing is these contexts come up all the time in real life. A human can take responsibility, but a computer can't and a human that doesn't in detail know exactly how the computer is designed and will deal with different situations can't make a reasonable judgement about how far to trust the computer in such situations anyway.

Oh computers can make the risk/reward calculations, their criteria for risk and reward is wholly in the hands of who develops the logic.

and there in lies the problems for companies developing these systems. with current law they would be liable for any negative outcome. logs would be required to be kept and every decision would surely be pulled apart by whatever paid experts each side can find.

so without legislation, legislation the consumer must be very wary about, comes about limiting liability to such systems they will have to side with caution and even then every event is a lawsuit waiting to happen.

finally, would you or anyone you know trust a system which states it can do something you find adversely risky to actually do with you or your loved ones at risk? what would that take? it might simply be a decision of local values

I don't know; in many fields you're certainly right, but most of the autonomous vehicle field is software companies and I can't recall any instance of them facing a major liability.

So would we call it a loss of autonomy if a human occupant would only grant an override saying "Standing still is not an option!"? I guess you would want such a feature when there is no steering wheel, e.g. if there is a forest fire close behind you.

While a computer can't judge that situation and doesn't "care" if it is destroyed it very well can keep driving no matter what.

A forest fire is a really good challenge scenario. The threat isn't really anything to do with driving or road hazards as such, it's an out of context threat.

It's these out of context situations that are my long term issue with losing the steering wheel. A typical example would be arriving at a country fair and parking in a field. I want to park on the side of the field that's currently in sunlight, but over the next hour will be shaded by those trees, near the gate but not too near the toilets, but not next to that patch of wet ground.

What kind of UI does the car need to have for me to specify exactly how and where I want it to park, without me actually driving it? Frankly I think this is level 6+. Level 5 is usually described as a self driving taxi, but family vehicles are used for things no taxi would ever have to deal with.

As for parking in a field, I'd want it to drop me off closest to the entry and then head to wherever the site wanted it to park.

Reply on news.ycombinator.com