Hacker News Reader

Comment by llukas | Hacker News Reader

https://www.thedrive.com/tech/36649/watch-a-tesla-model-3-fa...

So let me rephrase what you just said, you want your life as a pedestrian to rely on software that is not verified by 3rd party in any way and only rely on good faith of manufacturer? Did it ever work well?

Also such manufacturer can update software literally everyday without any external regression testing?

I agree that most of what you mention won't happen but failure mode of AI cars would be much different and imho much worse as pretty impossible to predict.

View on HN

Human failure models are already impossible to predict, but the impact of those failures are fairly easy to predict. Which is why there is so much pedestrian safety built into so many cities. You don’t need to know why that car is plowing full-speed into a pedestrian-filled NYC sidewalk (terrorism, medical emergency, human error, etc) because no matter what those failures are prevented the same way: giant blocks of concrete making it impossible for a car to get onto the sidewalk. Likewise with driving too fast, driving too slow, failing to stop, stopping unnecessarily, etc.

Any way an AI-driven car can fail, humans have been failing in the same way for a hundred years. Cars, by virtue of their incredibly limited mobility, can only fail in so many different ways. It’s not like a Tesla with autopilot can suddenly jump over barriers or drive sideways.

I think a Tesla where the terrorist has reprogrammed the autopilot can get through the barricades at high speeds, while a human could not. (most barricades I've seen are just fast enough apart that small maintenance trucks can get between them, but close enough that humans go slow - obviously not all are)

One example of a bad outcome is enough for you to decide that driverless cars are unacceptably dangerous? Take a look at number of collisions per mile driven and reconsider.

Collisions per mile driven? Autopilot disengages whenever there is something it cannot handle. How is that meaningful?

You could say it is "one example" if this was a human driver running over a pedestrian. If one AI car has a bug all models with same AI have it. There is 0.5M Model 3 and counting.

I just used one example as a point and it is much less important than issue of 3rd party verification. You don't know if manufacturer fixes the bug, when it is done and you never know if it regresses.

> I just used one example as a point and it is much less important than issue of 3rd party verification. You don't know if manufacturer fixes the bug, when it is done and you never know if it regresses.

No need to preemptively audit the code, accident statistics will clearly indicate unsafe behavior and economic/insurance disincentives[0] will induce continual safety improvements. Either a car will be safe enough to operate or it won't -- remember ~40,000 people are killed by American drivers every year.

[0]The 'Fight Club' equation.

Reply on news.ycombinator.com