I've thought something similar for a while, and it's been long enough that I think the problem is the failure mode. It might be that getting a car to recognize when it's encountered an edge case and needs to kick it to a human is about as challenging as just getting the car to come to the right conclusion and drive itself anyway. Intuitively we think of the problem as what the car needs to do, but the main problem is ascertaining the state of the road accurately. In the autonomous crashes we've seen from Tesla and Uber, it looks like the cars never even noticed enough to have an ambiguous condition they could deactivate autopilot from. They just had a completely erroneous perception of the outside world.