Hacker News Reader

Comment by dkonofalski | Hacker News Reader

This is why I don't understand the sad conclusion of the story in the article. The driver knew that AutoPilot had issues with that stretch of the freeway and was able to repeat the issues on several occasions but still thought it best to be on his phone, while driving 71 mph, at the mercy of AutoPilot? Especially considering that he was an engineer who had expressed concern over this, it seems silly to me that he wouldn't just take over manually during that stretch... He didn't and now he's gone because of it. It's not worth the risk/reward.

View on HN

We don't need to understand it. We just need to know that, even armed with all that knowledge, he still made the decision he did, and that we should not expect others with less knowledge of how software works to be any less human.

This has been my takeaway from the whole situation. It says something about the exactly how safe this stuff needs to be for it to be considered reliable.

My impression is that it's already more reliable than humans. Not perfect, but it probably averages better than all the crashes humans are causing.

> it's already more reliable than humans

Only when comparing aggregate statistics. Not all humans are equally (un)safe drivers; insurance rates vary based on driving record for good reason.

If anything I would expect less knowledgeable people to be more skeptical and cautious than somebody who has a vested interest and passion for technology. I can't imagine anybody other than a tech enthusiast who would maintain trust in a self driving car technology that previously attempted to veer them into a barrier.

100% agree with this. Most people I’ve talked to about this incident think the guy is an idiot; if you had found a car’s feature to be unsafe on a portion of your commute before, why on earth would you trust it with your and other’s lives? If you had a new fancy belaying device slip in the gym, you wouldn’t use it on your next big wall climb.

I also don’t get how everyone is forgiving him for being on his phone, in a construction zone no less. Reckless driving is reckless driving, being an Apple Engineer and Tesla owner doesn’t somehow negate that he was being a belligerent driver.

It's not about forgiving him for being an idiot. It's more about recognizing that there's more than just the one idiot on the road. To an approximation, pretty much everyone who has ever had a driver's license has been guilty of making an idiotic decision or two while behind the wheel.

And, Autopilot being a technology that, among other things, enables (or even encourages) idiotic behavior, there's real risk that placing too much blame on the driver's choices lulls us into an attitude that enables the next idiot to kill themselves and/or someone else.

I agree that everyone makes bad choices from time to time on the road. But as a society we put 100% of the blame on the driver if they aren’t doing everything in there power to prevent an accident. “Everyone makes mistakes” isn’t a get out of jail free card, booze clouds people’s judgement to the point where after enough drinks they’ll think they are good to drive, but that doesn’t change anything if they get behind the wheel, its still a DUI.

This guy was on his phone in a construction zone and crashed because of his lack of intervention, using a driving assist feature doesn’t somehow absolve him from being so preoccupied that his vehicle veered off the road and into a wall. Imagine if instead of him dying he had killed a construction worker; I have no doubt a jury would find him guilty of manslaughter. When you get into the driver’s seat of a car you are taking on responsibility for a death machine. I find it troubling that this conversation is happening at all, the blame should be put squarely on his shoulders.

Iff the car suddenly slammed the wheel to the side causing him to lose control or became unresponsive to his inputs, that would be another matter. But this could have been prevented if he wasn’t being grossly negligent of the risk he was partaking in behind the wheel.

An ounce of prevention is worth many, many pounds of assigning blame after the fact.

(Unless you're plaintiff's counsel, of course.)

> We just need to know that, even armed with all that knowledge, he still made the decision he did, and that we should not expect others with less knowledge of how software works to be any less human.

This is maybe one of the most important lessons of the 20th and 21st centuries (at least thus far): knowledge does not automatically prevent us from errors in judgments nor does it necessarily protect us from misfortune.

Nor does it absolve others from liability.

knowledge does not automatically prevent us from errors in judgments nor does it necessarily protect us from misfortune.

Relevant xkcd https://xkcd.com/795/

Not automatically, but it can help.

I was in this very situation at the top of Pikes Peak. A storm moved in and the park rangers closed the place and sent everybody away. They knew the statistics of being hit by lightning in that very place.

My friends like to talk about how awesome it would be to time travel or know what the outcome of a future decision will be. Even when armed with knowledge from observed outcomes of near-identical setup scenarios, majority of the time they end up doing exactly what they would have normally done. Happens to me from time to time.

Maybe we should keep it the way and use it as a culling method.

That’s the part I don’t get either. The only thing I can come up with is he wasn’t paying attention to the drive up to that point and didn’t notice he was in a problem area.

I have a Tesla and there are definitely problem areas. You learn them fairly quickly when you are taking the same route all the time and you’re trained to either turn off autopilot or at least be alert when going through those areas. Or maybe you test it out with your hands on the wheel ready to take over to see if they fixed the bug.

There were a couple spots on my normal driving routes where the car would inexplicably swerve. It happened one time in each spot and that was enough. Both those spots have been fixed since, but there’s no way I’d be on my phone not paying attention driving through there. I’m still cautious. There are two more spots where the car will brake to 45 mph on the highway and then speed back up after a few hundred feet. I am always on high alert around there and usually won’t even use autopilot in those areas.

> The only thing I can come up with is he wasn’t paying attention to the drive up to that point and didn’t notice he was in a problem area.

It's a well known phenomenon that the more you automate away routine tasks, the harder it is for the driver to stay alert and take over in non-routine situations. My understanding is that airplane pilots are specifically trained in strategies to avoid falling into that trap.

Tesla should pay for time spent in autopilot in these cases. It’s clearly not a finished product. That could incentivize attention paying as well.

> Or maybe you test it out with your hands on the wheel ready to take over to see if they fixed the bug.

Isn't holding the wheel always required now in Tesla's autopilot?

You have some flexibility to take your hands off the wheel for a bit. It nags you after a little bit and then gives you some more time to hold the wheel. If you keep your hands off for too long it will disengage and you can’t put autopilot back on until you put the car in park. You can satisfy the sensors by softly resting one hand on the wheel.

His phone was using data, that’s not proof he was using it and distracted. Spotify streaming a new playlist could be responsible for that.

> Recovered phone logs show that a strategy game called “Three Kingdoms” was “active during the driver’s trip to work,” the NTSB said. Investigators said the log data “does not provide enough information to ascertain” whether Huang “was holding the phone or how interactive he was,” though it said “most players have both hands on the phone to support the device and manipulate game actions.” Huang’s data usage was “consistent” with online game activity “about the time of the crash,” according to the NTSB.

Maybe he wasn't using his phone but he was distracted or maybe even asleep. Otherwise why let the car crash?

> ...but still thought it best to be on his phone, while driving 71 mph, at the mercy of AutoPilot?

> Records from an iPhone recovered from the crash site showed that Huang may have been using it before the accident. Records obtained from AT&T showed that data had been used while the vehicle was in motion, but the source of the transmissions couldn't be determined, the NTSB wrote. One transmission was less than one minute before the crash.

For all we know, that could mean he had spotify on.

Anyhow, "he was worried about it" is no reason to shift the blame to him.

I disagree. He was informed enough about the technology and aware enough of its limitations to write extensively about it. Tesla and all other assisted driving technologies all give the user a warning to stay alert and maintain control of the vehicle. He was clearly distracted somehow because he was relying on the technology.

Also:

He was an engineer - he probably wanted to try and "solve" the problem (isolate the car's behavior and see what inputs are causing the incorrect reaction) and engineer a solution. As an engineer, I can understand this drive. He probably felt he was on top of it because he knew of the bug.

I can understand that drive too. I cannot understand allowing a technology that you know to be unsafe take over completely so that I can play a game on my phone.

Why was Autopilot driving above the speed limit?

Or does Tesla allow you to set the speed?

It allows you to set the speed but only up to 8 mph above the posted speed limit (or something like that).

As far as I know my model 3 at least doesn't limit the auto-pilot speed other than the max which is 90MPH. It's just a scroll wheel, you can set it however you like.

I never used FSD but AP never let me go above 55mph in a 45mph zone when it was using traffic-aware driving.

Maybe he was fulfilling Apple's demands: either attend a meeting on the phone while driving, or wake up earlier.

(Definitely not singling out Apple here.. at IBM I had a coworker who was in an accident during a phone meeting- luckily non-fatal).

I have coworkers who regularly take meetings or text chat/email while driving. It makes me uncomfortable. I have tried hinting that it's a bad idea, but don't feel comfortable telling them directly (don't think it would do anything except harm our relationship due to me questioning their judgement) or reporting to HR (fear of retaliation, some of the coworkers are senior to me in the management chain). On the other hand my discomfort kind of makes me feel like a busybody. 99 times out of 100 nothing will happen and the 100th time will probably just be a fenderbender. Not sure what to do except continue feeling uncomfortable with it.

"Are you driving right now? We can reschedule to something more convenient for you? I don't want you to get pulled over."

You're way too cautious in your human interactions, which is terribly sad. But I understand it because of our business culture.

We had this PM who ran our meetings, when people phoned in from the road she would say, "please take your calls while your aren't driving" and boot them from the meeting. Now I do the same thing and you should to.

I didn't see that in the original article, but I do see it here:

https://www.washingtonpost.com/transportation/2020/02/11/tel...

Ah, I read the wp article yesterday and just assumed it was the same article being discussed here.

It was a mistake, people make them all the time. Mistakes in cars kill people, and you don't need to be driving a Tesla to see that.

Frankly the whole point of automated control is to reduce this kind of mistake fatality, and... I mean, it's working. This was a tragedy for sure, but it was also fully two years ago. These events haven't been recurring, it's likely the specific proximate causes have been addressed, and by all reckoning these systems (while still not flawless!) are at or above parity with alert human drivers in terms of safety.

Basically, I read the same facts you do and take the opposite conclusion. People make bad risk/reward decisions all the time, so we need to take them out of the loop to the extent practical.

Reply on news.ycombinator.com