The things that the car needs to know are largely standardized: the lines on the road, road signs, speed limits, etc.
These are things humans need to know as well, and yet autonomous car cheerleaders constantly argue that human drivers are death machines, despite the fact that most human drivers know perfectly well how to follow these norms, and even deviate from them, without incident, most of the time.
I suspect that in their enthusiasm to set the bar of human intelligence as low as possible in order to make the case for autonomous cars seem urgent and necessary, some vastly underestimate the actual complexity of the problem. An autonomous car that only knows to 'avoid the boxes' has worse AI than many modern video games.