That’s also why people tend to shy away from people with mental health issues. We don’t have a good theory of mind to intuit their behavior.
Fact is, humans in general are imperfect operators. The ai driver only has to be an alright driver, not a perfect one, to route around the long tail of drivers that cause most all the fatalities.
I think you missed my point. Because software is more opaque, it has a much higher threshold before the public feels comfortable with it. My claim is it will have to be an outstanding driver, not just an “alright” one before autonomous driving is given the reins en masse. In addition, I don’t think we know much about the true distribution of risk, so claims about the long-tail are undefined and somewhat meaningless. We don’t have a codified “edge” that defines what you call edge cases. Both software and people are imperfect. Given the opaqueness of software, I still maintain people are more comfortable with human drivers due to the evolved theory of mind. Do you think more people would prefer their non-seatbelted toddler to be in a an average autonomous vehicle by themselves or with a human driver give the current state of the art?
But more to my point, humans are also irrational so statistical arguments don’t translate well to policy. Just look at how many people (and professionals) trade unleveraged stocks when an index fund is a better statistical bet. Your point hinges on humans being modeled as rational actors and my point is that is a bad assumption. It’s a sociological problem as much as an engineering one.
Fixing that would require "full self stopping". Coming soon[1].
[1] ... for some value of "soon", that is.