Comment by robbrown451 | Hacker News Reader

Comment by robbrown451 | Hacker News Reader

Parent Full thread

"But it does not change that the autonomous machine does not understand the complex world it is driving in in the slightest, and that, for example, swerving into an unmanned fruit stand without being able to brake is much better than swerving into an unmanned gas pump."

It seems it would have a way of prioritizing such things. That doesn't seem particularly complicated, to be honest... weighted decision making is certainly within its capacity. E.g. shopping cart has fewer "avoid hitting" points than stroller.

It's not like every scenario has to be explicitly programmed in, nor does the program need to run some analysis on a detailed backstory to justify that a baby is more valuable than groceries. In effect, somebody -- probably not a programmer either -- just needs to enter some numbers into a spreadsheet.

(yes there is complex programming that allows that to be manifested in the car's decisions, but the idea that programmers are themselves constantly making "moral calls" in the code, rather than the control data, is fiction)

And if it does have such prioritization in its logic, I'd say yeah, it "understands" the world in that respect. Unless you have defined the word "understand" in some mystical way that precludes non-biological machines by definition.

> It seems it would have a way of prioritizing such things.

You are putting the cart before the horse. The problem is not in prioritization, the problem is in having the correct ontology to even get to the "prioritization" stage.

Does the car know what a fruit stand is? Does it know what a gas pump is? Does it know how the fruit stand relates to the gas pump in "expected outcome when being hit by a car"?

If you say "we can program that in", read my post again.

I would imagine the car has ways of identifying things that are specific hazards, such as a gas pump. Fruit stand is probably categorized as "other."

"the problem is in having the correct ontology to even get to the "prioritization" stage."

That part isn't done by the program, it is done by whoever enters the prioritization numbers. That is, someone, possibly a committee, can dial up the "avoid gas pumps" weighting relative to the "avoid baby stroller" weighting if they are concerned that cars might swerve so widely to avoid coming near a stroller than they are risking hitting a different hazard. Or they can dial up the weight of grocery carts relative to dogs, since children might be in a grocery cart. Etc.

Those are humans, who can do whatever ontological analysis they need when deciding on the the settings. The car doesn't need to access any of that, it just needs a general look up table that can help make optimal decisions based on the human-entered value system.

Sure, we'll just write down all the things there are in the world in order of how bad it is for a car to hit them.

We're really lucky we never see more than 1 thing at a time on the road.

You don't think self driving cars categorize and prioritize things?

I mean, you're right; someone making that list might not think to include "centaur", and maybe one Halloween a child is dressed up as one, and the computer vision system interprets the "centaur" as a horse instead of a child, and it makes the wrong decision, but how many centaur-related accidents do you think self-driving cars are going to be involved in each year?

It's completely feasible to imagine writing a list of the top, say, 100 things in the world that a car needs to make morally-significant decisions about, and then deal with every other accident or near-miss after the fact. Interactions with unrecognised objects should be rare enough to be a rounding error when comparing accident rates between autonomous and human-driven miles.

Even if a automated vehicles intelligence misinterpreted a human child to be a horse child, then it should only hit it in the unavoidable circumstance of trying to preserve a human life.

If it’s choosing between hitting a pole and hitting a pony, it should always hit the pole so long as no one is injured.

The real problem is that occasionally these cars today mistake roads for oceans, walls for roads, and people for inanimate poles.

> Does the car know what a **** is?

On a level below identifying stop signs and lollipop ladies and push carts, a SDC's stack needs to be able to identify:

1) Driveable areas. If something looks like a cliff maybe don't go there.

2) Fleeting obstacles. Dust blowing in the wind. A stray plastic bag, winging its way northwards to the waiting maw of a baby turtle. A person with a borderline credit score. A stray cat chasing a bug. That sort of thing.

3) Anything else that's physically present in the path of the car. Doesn't matter what it is. Do Not Hit The Thing is the second lesson anyone learns when being taught to drive, after Make It Go So Hit The Thing Is Even An Option.

You're asking the car to make moral decisions. Given the choice between hitting a child or and old woman, which will it choose? A bicyclist vs a pedestrian? A lemonade vendor vs a hotdog vendor?

This is all assuming it can distinguish between all of these objects, and that a real person could assign relative moral values to hitting one over the other.

Reply on news.ycombinator.com