New technologies tend to be rather unsafe when introduced: steam engines, cars, trains, airplanes, electricity, gas heating, all those things tended to kill the early adopters and some bystanders as well. Then they improve and the next generations get used to them and their inherent risks that diminish over time but rarely get down to pure 0.
We are much more risk-aware than our ancestors, though, because we no longer live in a world of ubiquitous premature death. With an average life expectancy exceeding 80 in many countries, we tend to be much more careful than our ancestors in times when life expectancy was 47. (As in the USA in the year 1900.)
I can understand that - I do not want to die under wheels of a robot gone wild any more than anyone else - but we should still find a balance, unless we want to stagnate indefinitely.
The risks of self driving cars are nothing compared to the risks of biotech, which is just coming of age. But biotech holds a lot of promises, too.
PS.: Frankly, my worst fear about self-driving cars is malware and ransomware, not honest mistakes in the code. Computer security is basically an oxymoron and some people are smart and evil at the same time.