Hacker News Reader

Comment by brianwawok | Hacker News Reader

The driver is responsible for stopping the car. You literally agree to it when you enable AP. People are too comfortable and not paying attention.

View on HN

> The driver is responsible for stopping the car. You literally agree to it when you enable AP. People are too comfortable and not paying attention.

Autopilot encourages people to not pay as much attention while driving, until a critical situation comes up at which point they're expected to be fully in control. That's a design flaw that isn't fixed with a liability shifting "Agree" button.

It will nag you every 17 seconds (if not attentive/holding the wheel with a torque) and abort auto pilot completely if you do not comply.

If you do this three times in a drive you have to power off and on the car again to be able to use autopilot again.

I've done maybe 40.000 km on autopilot and it's very aggressive compared to my other car (ioniq electric) when it comes to attention.

The fact that some people purposely defeat this system with weights on the wheel is another matter.

But saying that tesla encourages you to not paying attention is not correct.

To be fair, the person you are responding to may not like those other systems either.

All the articles with industry analysts insisting that other driver monitoring systems are nearly perfect have sometimes blinded me to that fact.

A lot of people don't like any of these systems for a lot of the same reasons that people didn't like basic cruise control in the early days.

I haven't used basic cruise control in years. It's actually convenient not to have to pay attention to the speed but I find, on roads with even moderate traffic, it comes at the cost of prioritizing maintaining a constant speed rather than flowing with traffic and generally staying in a lane.

The fact that it occurs so often should suggest to you that autopilot itself makes people too comfortable and not pay attention.

I, and many others, take the position that there really isn't any practical gradation between more-or-less conventional driving, maybe with light lane switching assistance or something, and full on "Level 5" autonomous driving, because the idea that human brains can watch something for hours at a time only to take instant, sub-second actions every few hours is an insane model of human cognition.

I don't disagree with the idea of having a gradient to measure our progress up to full autonomy, but I haven't seen much to say that we should deploy the intermediate stages. Until autonomy can be reasonably guaranteed at at least human levels of safety under the assumption of no human control, it really shouldn't be deployed. "But the human had 673 milliseconds to take control and avert the accident! It's their fault!" is simply not a reasonable defense.

Shouldn't the objection be to anything less than Level 4, rather than Level 5? Level 4 requires that the vehicle can safely stop itself and wait for a human to take over when it is faced with a situation it can't handle.

(There's an argument that if you take away the routine parts of driving, humans won't be able to deal with anomalous situations, but that's a different argument than the attentiveness one.)

If that is possible, yes. I'm not convinced it is possible to create an autonomous vehicle that only panics in situations where there is a safe pull-over, or, if such a vehicle exists, that it'll be considered "autonomous" when it can't go past 25 or 35 mph or something. There's plenty of driving situations that may be confusing where pulling over and stopping isn't possible.

(And, to be clear, I'm not asking for perfect safety, I'm asking for "human or better", and also understand that there may be some interesting fuzziness near the border where the two have different strengths and weakenesses but come out statistically the same. However, I don't feel like we're there yet.)

Mixed control between automated and human for tasks that require persistent attention is bad. We have known this for decades. The example often given in engineering textbooks is 3 mile island. I suspect that the engineers involved in self driving systems know this too, but the liability is too great to make the switch all at once. It seems better to take baby steps into self driving, but history teaches us it should be all or nothing.

You could say the same about airplane autopilots, but they have an amazing safety record.

The airline autopilot doesn't have to negotiate anything like the kind of hazards that a car driving system does. Also, pilot training, the presence of a whole flight crew to monitor and catch errors make for a completely different situation.

Not to mention you have to be licensed and spend hundreds of hours to share controls of a commercial plane.

Car AP is way harder than plane AP and saying "But planes can do it, why is Tesla so bad at this" is facetious.

Far fewer objects to hit in the air: disengagement means "keep going this way," rarely "and crash into an eventual obstacle". Plus, their early safety record is written in blood like everything else is.

> Plus, their early safety record is written in blood like everything else is.

Yeah, and that's how it's going to be with car autopilots. Which is why it pisses me off that people want to stop progress on it.

Interesting typoes in "be unwilling test subjects". Feel free to volunteer, though. Elaine Herzberg sure didn't.

They have their own records of fuck ups [0]. And for similar reasons as the Tesla AP crashes. Autopilot disengages and leaves the pilot in the lurch. If the pilot doesn't have situational awareness right away it can be fatal.

Airplane autopilots, depending on generation, are full touch. The pilot programs the waypoints. The pilot is alerted and agrees to every course correction and change.

[0] https://en.wikipedia.org/wiki/Air_France_Flight_447

Airplane autopilots don't disengage. They do what you told them to do. They are simple and predictable systems that at most simply follow a predetermined course exactly.

That's not entirely true, there are situations where they do disengage automatically (e.g. due to missing/conflicting sensor information). By the nature of the thing it is a) rare and b) the pilots have more time to react because it won't be a "you're going to hit a truck in 1 second" situation unless something went badly wrong before. But there is still some risk to the handover situation making it more likely pilots misinterpret the current state of the plane or do not realize which automatic support is still active (not just autopilot, also other avionics support (e.g. if I remember correctly one aspect around the Air France 447 crash is that the pilot likely did not realize that the plane had disengaged support systems that would keep him inside a safe flight envelope in normal circumstances).

Airplanes also have 2 pilots and nothing to crash into.

You have no telemetry and no idea what actually happened, car might have veered without enough warning. Which is the whole point of having an investigation, really.

Reply on news.ycombinator.com