Hacker News Reader

Comment by borski | Hacker News Reader

> The night example looks to be specifically stacked against autopilot.

I would argue so is the real world.

It is exactly the same scenario where Uber self-driving killed a pedestrian crossing the road at night

The exact situation where "uber self driving" killed a pedestrian was: the driver was literally watching a movie at her job, while she was supposed to be driving a car and training a self driving system.

A driver killed a pedestrian because she wasn't paying attention: https://www.youtube.com/watch?v=hthyTh_fopo

Sure, but this was supposed to be fully autonomous. Nobody is arguing the human didn’t make a mistake. The autonomous system, however, definitely also did.

Wasn't this not a pedestrian, but a cyclist crossing in a completely inappropriate place? Granted an SDC should still react to this while many humans in the same situation would not.

Pedestrian slowly walking a bike, with basically no reflective clothing, on a dark night. This is exactly how humans kill pedestrians with cars all the time.

Sure, but that's also why that specific car was equipped with a radar-based obstacle detection.....which the company specifically disabled. There's a very good chance that this system would have saved that person's life. Also while yes, humans are crap at this, it's very rare that you'd just plow into someone at full speed without even attempting to slow down or swerve - which is exactly what the car did.

This may be technically true (I actually don't know what the drivers full intended purpose at the time was) but it doesn't negate some extremely sketchy software practices on a safety critical system, like "action suppression" to avoid nuisance braking.

As in most accidents of this nature, there is a chain of mistakes. It's bad practice to ignore some mistakes simply because we can also point to other failures in the chain of events.

Volvo's emergency braking system, which detected it and would have braked in time, had been restricted by Uber to not be able to take any action.

Uber's system was set in a way that "non identified object in my way" didn't trigger an immediate slow down, but instead a "sleep it off for a second and check again". It saw the impact coming, and instead of notifying the driver it decided to wait a full second first, because it was programmed to do so. Which any programmer can recognize as the "turn it off and on again" idiom that tells us their system was misidentifying lots of things.

What the driver did or did not do once notified doesn't change that. That car would have killed someone, somewhere, sometime, because it was programmed to not avoid it.

You mean when the uber test car detected the person but was programmed to do nothing when an obstruction was detected

I'm fine doing away with Uber's self-driving as well. Although I think Tesla's is the worst of the lot, I'm not confident in or thrilled by any self-driving tech on public roads in the next decade

Reply on news.ycombinator.com