I think you misinterpret what I said, it is the edge case scenarios where fatalities tend to occur. Of course in humans that can be due to drunkeness or distraction or confusion, nobody is claiming human drivers are perfect, but if it is the 'weird' situations that tend to result in accidents and deep learning tends to make mistakes in 'weird' circumstances, that doesn't fill me with confidence.

Also it is not at all obvious that self driving cars will do better than humans. Perhaps they will, but it isn't just about numbers. As I alluded to in another comment, imagine if they had 50% of the human death rate but it was all small children and occurred seemingly randomly. Do you think that would be acceptable and a better outcome? These things are less black and white than they might seem.

Another factor is that with deep learning it is fully a black box. It is very difficult to understand WHY a particular course of action was taken, whereas in human cases you can usually determine this. The why might also be limited, e.g. 'human child was mislabelled as stop sign' or similar, which just causes further questions.

The fact that you have scenarios the car will definitely not know how to handle combined with a deep level of inscrutability makes it difficult to have confidence in issues being resolved or the technology improving on a safety basis.