Fatal Tesla Crash Raises New Questions About Autopilot System | Hacker News Reader

Fatal Tesla Crash Raises New Questions About Autopilot System

nytimes.com

8 pointsjijojv6 comments

How did the car not even apply the brakes? How could it fail that bad?

Maybe it's time for people to admit that self-driving is being romanticized by carmakers because they're trying to sell you their cars and therefore they have an incentive to blow their capabilities out of proportion.

I think for one carmakers are misleading about these systems' true capabilities, but second, I don't think they even know how imperfect these systems are because of a potentially unimaginable number of edge cases (which they thought would be much smaller and manageable).

There are three narratives I’ve come into contact with vis a vis SDV’s.

1.) Utopian: Any sacrifice is worth getting people out from behind the wheel. That sacrifice never seems to belong to the person espousing this view. I run into this almost exclusively from friends and acquaintances in tech, or online. It’s an extension of all utopian thinking, and like all utopian thinking is dangerous.

2.) The future is now: This argument (often yells) that SDV’s are currently safer than human drivers, so just shut up, people kill people too. When evidence to the contrary is presented they frequently reveal themselves to be utopian.

3.) Wait, what?: People who have no idea just how widespread this technology is on the roads, and range from uneasy to terrified. I mostly run across this from people I know who aren’t in tech, and rarely see this online.

When I show people in group 3 comments from groups 1 and 2, they get really really pissed.

This is the best taxonomy I have seen. I'd like to add another, smaller category.

There is a fourth group, people in tech, who are fully aware how widespread this technology is on the roads and follow it like a train wreck in slow motion.

  they're trying to sell you their cars

... and SDVs open a whole new market to them: non-drivers.

Why are you surprised?

At this point, it is a well-known failure mode of the Tesla Autopilot:

It does not notice tractor trailers. It does not notice street sweepers. It does not notice fire trucks. It does not notice lane barriers.

All happened in daylight, and in good conditions. The Autopilot just continues in high speed without even attempting to slow down - Controlled Flight Into Terrain.

Wired has a piece trying to explain why it cannot notice stationary objects [1].

It's like the joke about the dancing bear. We are impressed not by how well the bear dances, but by the fact it can dance at all.

[1] https://www.wired.com/story/tesla-autopilot-why-crash-radar/

Reply on news.ycombinator.com