When cars were introduced, people kept getting hit in the street. "Jaywalking" as a crime was created to shift the blame away from the technology (cars) to the victims (pedestrians).
For self-driving vehicles, it suffices that we outlaw the edge cases, and enforce it to make them insignificant. The Uber ATC crash? Blame the jaywalker. Kid chasing a ball in the street? Blame the parents. Treat self-driving cars the same way that we treat trains: get out of the way, or you've no one to blame but yourself.
What that leaves practically is preventing crashes with the static environment, self-driving cars, and "normal" road users (i.e. ones that can't be blamed if a collision occurs). This is much easier than the plastic bags and animals and cyclists and jaywalkers and wacky drivers that form most of the remaining unhandled 1%.
The only country that can pull this off, IMO, is China. The mix of authoritarianism plus technical prowess allows the missing 1% of cases to be bulldozed in the name of progress.