http://99percentinvisible.org/episode/children-of-the-magent...
With self driving cars where we're relying on the car for moment-to-moment instantaneous reactions, I think it's very unlikely that human drivers will be able to pick up the slack in a system failure. If you're expecting the car to stop it self and it doesn't, you have what, a quarter second to jump in manually?
The recent video of a self driving uber going through a red light is probably a similar scenario. Even though uber insists it's a "driver assistance technology" and not a self driving car, the driver expected the car to stop itself, so they didn't bother applying the brakes. And that one was at low speeds, maybe 10 mph.
The saving grace here is that they're competing against human drivers. The technology doesn't have to be perfect, just better than us. But relying on human oversight to maintain 100% perfect attention for a six hour journey of doing literally nothing but watching a car drive itself in case you need to step in? I think that's a non-starter. For these to work, the car has to be able to manage on its own.
I imagine this has a similar effect.
The flip side is, I always have my foot resting lightly on the gas, so I'm not really at a disadvantage if I need to brake. Do Tesla drivers using autopilot have their hands hovering over the wheel typically? If not I imagine reaction time could be impacted.
In general, I hypothesise that the more a system is automated, the less awareness is demanded of the operator and thus the more likely the operator is to get into some kind of trouble, whether this stems from development of a lazy habit, rustiness or just not being able to recover the situation if the automation abruptly fails. In fact, we've seen several plane crashes attributed to pilot error related to increasing automation in aircraft.