This is a false anthropomorphization. Yes, it would be weird if a human were in this place, but computers are not humans. They can do things we could never do, and things that are straightforward for us like reading a text CAPTCHA, are borderline impossible sometimes. Still, my phone is way better at remembering the story of my life than I am.
If you mean "it's not reasonable to hold all drivers, whether humans or AI, to the standards required to pass a driving test (left turns, sharing the road with bicycles and construction workers, rain)", then ... uh, I dunno what to tell you.
And they will be: accidents per mile. Or some variation on that that takes into account severity of the accident. We already revoke driver licenses if there are too many incidents.
Having a car give up 30% of the time when it encounters an edge case, but driving better than 99% of the other humans the other 70% of the time, is still better driving than a single human... the same way a human speaking and reading books at age 7 is still more advanced than the most experienced, cutting-edge dog.
I can handle quite a lot, but I don't handle a lot of things that a self-driving car can like 360 degree vision at all times or lossless memory of millions of past trips. Nor can any other human.