Hacker News Reader

Comment by codingdave | Hacker News Reader

I personally am just missing one thing to give me confidence in self-driving cars - transparency into their algorithms. I don't need to get deep into them, but some general information on how they "think" for accident avoidance would help me.

Just as an example, lets assume some debris on the highway. I am sure the self-driving car can avoid it, or stop until there is a clear space to go around it. But what if a human is tailgating? Does the car ignore that? Or does it (facetiously anthropomorphically) think to itself, "Hey, that is a human behind me, tailgating, at 75 mph. Swerving left to avoid this debris would put the human-driven vehicle on a path to collide with it, because the poor sap doesn't have my reaction time. I'd better hit the brakes instead, to communicate the danger."

I know that is an edge case, but edge cases are where accidents happen anyway, so I'd like to know what to expect from self-driven cars.

View on HN

Edge cases are where accidents happen because humans suck at them, whereas you can specifically train on them with a computer.

The bigger problem, that you allude to, is that the algorithms that actually work are mostly machine learned and therefore very opaque to human inspection.

One of the big issues with this is how consumers/government will require algorithms to prioritize.

i.e. one can imagine that there are accidents with a certain context wherein a driverless car could prevent x amount of people from dying, by killing its own passenger. For example an axle breaks, you can't turn right anymore, only left, and you're coming up on a right turn. You're supposed to turn right and stick to your lane, you can't, you either go straight onto the other lane, or go left. You happen to be driving an SUV and there's two small cars in the lane. If you go straight you hit both of them, and the camera registers 2 young children in each and an adult driving. Given the difference in car sizes, you're likely to survive but also likely to kill at least two kids, perhaps up to 6 people, you'll definitely destroy 2 cars and injure all 6. Or you take a sharp left and drive infront of them straight off the road, you drop down in a cliff and die. In both cases the computer calculates to a large degree of certainty that if either choice is made that it'd go down pretty much like this.

Now this is a silly edge case but it's the best I could come up with right away, more plausible ones exists. Anyway the question now becomes, is it okay for cars to be sold that are programmed to prioritize the passenger's life, i.e. killing the maximum amount of innocent people who are not you, estimated between 2 and 6, injuring at least all 6, creating the biggest economic damage and loss of life or livelihood to innocent people (particularly 'innocent' as their car's axle was just fine, yours wasn't).

Or would a government require the algorithm to take a different priority, one in which the car's software is programmed to sacrifice the car's owner/passenger, for the sake of a utilitarian calculation?

I think the answer is pretty clear what's fair, and that if a computer wasn't part of the equation that an ordinary human would likely not have had the mental fortitude to identify the axle steering issue and all options and choose the best one and that the outcome would be worse than a driverless car. But I also think it's pretty clear that the 'fair' decision would still be highly controversial and scary to imagine and something that'll see quite a lot of debate and pockets of resistance the next few decades.

It somewhat annoys me that we could eliminate 99.9% of accidents, yet people still would object to a self-driving car because of contrived fictional situations like the one described above.

A freak situation like the one described above would surely make the news - "Hero driver saves family by giving his life" and/or "Our mom would still be alive if that asshole wasn't so selfish". Yet I don't remember seeing a single story like this, ever. From comment threads on self-driving vehicles, you'd think these moral dilemmas are a daily, if not an hourly, thing in traffic.

Don't take discussion as objection. If it was up to me we'd go driverless asap.

In almost any case I can think of, the answer is: slam on the brakes and come to a complete stop.

Do you allow that situations exist (in the billions of miles driven by the world population every day) in which a driver, whether human or computer, can be faced with a situation where different actions are possible, and one sacrifices the passenger's life but prevents harm to other nearby cars, and another action would harm nearby cars but prevent harm to the passenger of the car, and that one of these actions can be 'slam on the breaks'? And that thus there's a trade off by your car's software between your life and someone else's?

Even if that trade off comes from 100 different actions that can be taken, that situations exist in which someone inevitably gets harmed and the different decisions distribute that harm either to you, or passengers of other cars?

Whatever that situation may be, do you allow that it can exist, even if it's extremely rare, and that someone has to make a decision on how to set the computer's priorities in such an edge case?

That's the point I'm making. The specific example of such a situation isn't very important (unless you don't acknowledge ANY situation with such a tradeoff decision could ever to occur in the over 4000 billion km driven ever year which I'd say is pretty unlikely).

I would say that in all cases, any decision taken by the computer would probably be miles better than a decision made by a human driver in such a hypothetical situation. Firstly, the computer will have a much better overview of what's going on than any hman would. Secondly, a human is most likely to just panic and slam on the brakes as hard as possible.

The problem with hypotheticals is just that: they're hypothetical. They don't exist in the real world. There's never a situation where you have precisely 2 options, one of which kills your passenger, the other of which kills oncoming cars. There's always a multitude of options, with various risk factors associated with each one.

In that example the computer would brake the two wheels in the right side, and apply torque on the two outer wheels, so that the car turns.

While human drivers are limited to 2 or 3 pedals and a steering wheel, a driverless car can control a lot of elements individually.

> is that the algorithms that actually work are mostly machine learned and therefore very opaque to human inspection.

The identification of objects is, but the actual driving logic isn't

You still have a list of 'check if light is green', 'check there are no obstacles', 'accelerate slowly' etc

Any source? There isn't much on the web about self driving car software architectures.

You can totally train for those three scenarios (well, in a simulated environment anyways), but whether they have the trust or tech to do that yet, I don't know.

I've been driving for a few weeks with Waze running most of the time and the human generated alerts are incredibly useful.

Accident ahead. Object on road ahead. Car stopped on shoulder ahead.

There will be a way for self driving cars to broadcast these types of alerts for human drivers. I would expect that the self driving car would attempt to preserve itself (and prevent harm to its occupants) before trying to protect a human driver from harm.

In America, automatic cars need automatic weapons to threaten tailgaters, just like normal cars with human drivers have.

Reply on news.ycombinator.com