Perhaps because the promise of autonomy is that you will be able to let go, close your eyes, take a nap, or read a book. That is really hard to do if you have even a 0.0001% chance of hitting something and dying because your car didn't do something right.

Sure, your chances of getting into an accident while driving your own car are higher, and some dummy could throw a turkey onto your windshield and kill you ...etc, but humans don't think in probabilities (well, outside of Hacker News humans), and while driving their on car humans have an illusion of control, when the car is driving itself and you are sleeping, that control is gone.