Once autonomous cars have saturated the marketplace, I can only think of two reasons to still drive manually:
1. As with people who row or sail, just for the enjoyment. Such usage could be restricted to specific places like racetracks.
2. Because I'm going somewhere I "shouldn't" and I don't want to tell my car (and have it record my journey for someone else to review, or refuse to take me). This may also be the case with emergency vehicles, an ambulance may need to drive along a pedestrian-only street, for example.
Now, there will be a need to mix human vs autonomous traffic on the run-up to that. Obviously, until sufficiently many cars are autonomous to warrant the removal of human-controlled cars, that will need to be the case. Furthermore, early production Autonomous cars may have routefinding flaws, or the computer may break down, meaning that someone may need to drive a normally autonomous car themselves.