I'm the opposite. I don't think I'll buy another car (current is 3 years old) and I'm even surer that my son (aged 2) is not going to learn how to drive.
Driving is a pretty well defined space. Stay in your lane, get to the destination, and don't hit anyone. Pretty much every similar problem that significant effort has been solved. Chess, Go, Starcraft, DOTA all were called unsolvable at one point and all were eventually solved.
Your edge cases are just that. Edge cases. Nobody is going to care if they can't drive in extremely heavy snow. To begin with, they probably shouldn't do that anyways. More importantly the vast benefits of self driving cars outweighs the inconvenience of not being able to drive a dozen times a year. We all still have cell phones even though you can't make calls in elevators.
Even then, if you can do it why can't a computer? Rocking the car back and forth seems doable. Following an unpaved road is much harder, but then again are humans actually good at it either?
You might be right that they will still have steering wheels for no other reason than there's little downside to including them.