1. Be robust to bad visibility, not an easy task in computer vision.
2. Have access to extremely detailed maps that include things like driveways and parking lots. These maps would need to be continuously updated. The labor required to make and maintain such maps will limit how many places cars can be fully autonomous.
3. Understand the etiquette of the road. For example, a person seeing a car stop in the street near a parking spot would assume the car is about to parallel park and give it room to pull off the maneuver. An autonomous vehicle wouldn't give it so much room, because it can't reason about context in that way.
4. Negotiate ambiguous situations requiring interaction with other drivers. There are frequently situations where it's not clear who has the right of way from the rules of the road. People resolve these by gesturing with their hands and with their cars. A fully autonomous vehicle would need to understand these signals.