Hacker News Reader

Comment by autoexec | Hacker News Reader

...because the self-driving feature turned itself off after detecting the crash in the fractions of a millisecond before the crash was recorded

View on HN

In a rare bit of still-sane news, the US National Highway and Safety Administration staff aren't dumb: Their policy is to consider whether any automation was active in 30 seconds before the crash.

https://www.nhtsa.gov/laws-regulations/standing-general-orde...

the whole concept of having cars with that are almost capable of self-driving is utterly insane, the very least you'd need special training. We are not equipped to deal with something that works brilliantly most of the times but might kill you for no foreseeable reason.

Yeah, that's definitely the frustrating zeitgeist for me these days, which extends to LLMs.

I'd also emphasize that it's not just error rate, but the shape/distribution of errors, and our (in)ability to build control systems around them.

To illustrate, imagine if someone unveiled a car which was unambiguously safer in every statistical measure... buuuut some of its unsafety came from jumping the curb to chase and kill pedestrians, under circumstances we can't predict for reasons we can't diagnose.

Even pilots are getting this automation-complacency now. A plane crashed because the military jammed GPS and the pilot wasn't used to any backup procedures even though 30 years ago they were the only procedures.

How many million dollars do you think will reverse this policy?

"Uh oh, this is a problem. Welp, I'm outta here, good luck."

Reply on news.ycombinator.com