You're missing the point. "Car deaths happen constantly" isn't a good enough rebut. People will always think they would have been able to avoid the accident if they were in control.

It's worse because so far the few really bad accidents caused by "self driving" cars would have been stopped if a human was in control (see: Tesla + truck accident).