Hacker News Reader

Comment by jjulius | Hacker News Reader

>But that said, once you learn the quirks it is extremely predictable and robust.

I feel like a self-driving vehicle is one of those areas where nobody should have to "learn the quirks".

It's honestly like a black mirror episode. "I have to have maintain a mental model of how my black box ML car AI will act so I don't die or kill other road users... and it can be updated daily."

Autopilot in aircraft have a similar problem, which led to the Air France Flight 447 crash in 2009. The airspeed indicators froze over, the autopilot switched from "normal law" to "alternate law", and the pilots failed to understand that the autopilot's protection from stalling no longer applied. They did not have enough training (or were too confused) to fly without the help of automated aids and flew the plane right into the ocean.

Boeing's planes were known for having fewer of these automated aids and thus required more skill to fly the plane, but the 747-MAX debacle threw cold water on that idea.

Of course, (unlike some commenters in this thread) the air travel industry thankfully had the good sense to look at autopilot on balance rather than fixating on a single fatal accident (or in the case of Tesla's Autopilot, the mere possibility of a fatal accident). The salient question isn't whether or not an autopilot allows for any fatalities, but rather whether or not it reduces fatalities relative to manual piloting/driving.

Yep, autopilot is a good helper but a bad master. In the air there's a lot more space and fewer objects to avoid than on the roads, yet nobody has managed to build a perfect autopilot that could really replace pilots in every situation. I remain skeptical that we'll ever see one in any dimension, let it be air, sea or land.

> "I have to have maintain a mental model of how my black box ML car AI will act so I don't die or kill other road users... and it can be updated daily."

Not defending Tesla here, but this is exactly how I think of other drivers on the road, whether I'm driving, cycling or walking.

Yes, we are literally trained for that - to keep distance with cars ahead of us, expect the unexpected and so on. We are also told not to drive cars without proper maintaince (and inforced by law in most places) so our cars are reliable.

I wonder if future driver training will factor in spotting problems with self-driving or related functions.

But Autopilot is not really a self driving vehicle though - it's basically just lane keeping + adaptive cruise control. It will gladly blow through a stop sign or a red light, it can't change lanes, or make turns, or do any other completely normal driving things that one would expect from a self driving car.

Navigate-on-Autopilot, and some other features which are part of the FSD package that is already publicly available, absolutely will stop at stop signs and lights (although the process is often annoying and buggy), change lanes (sometimes into a trailer if it doesn’t see it quite right), and make turns (takes exits/ramps), although these are things that can be individually toggled on and off. They’re there, but the implementation is kinda buggy shitware.

I'm pretty sure Tesla's with FSD get in less accidents when compared to the average driver. Mine tends to be frustratingly overly cautious in most cases.

Except when it suddenly turns into incoming traffic. Don’t believe stats from Tesla.

> Except when it suddenly turns into incoming traffic

Is there any reason to suggest these aren't factored into the aforementioned data?

> Don’t believe stats from Tesla.

I'm all for more transparency, but I'm certainly not going to trust a random Internet commenter either.

Source?

I guess the rest of us drivers have to "learn the quirks" the hard way?

If you’re not tailgating someone, you’ll be fine, and anyway “the rest of us drivers” includes plenty of people who drive much more dangerously than an overly cautious AI. Autopilot has many millions of miles of driving time and as far as I know there hasn’t been even a single fatal accident due to phantom braking.

Reply on news.ycombinator.com