The obvious counterpoint to "operating from stored data" is an accident or construction causing a lane to be closed, or a child chasing a ball across the road.
FWIW I am a near-term self-driving car skeptic and have been for a long time. I just think that these are not the kind of issues that really pose a major obstacle whereas drunk people wandering across the street are.