In some sense, the self-driving system gets to be trained with experiences of failure, probably under the expectation that no intervention will occur (i.e. the "policy" cannot include human intervention). But the human driver doesn't get to learn from experience when to not trust the self-driving system (because getting it wrong can be disastrous).
Perhaps a missing piece that human drivers _should_ participate in, but almost certainly won't, is training in a simulator. There's a methodological challenge here -- ideally the simulator training should focus on scenarios where the self-driving system is weakest (i.e. where it's plausible that the driver will need to intervene), but those are likely also cases where we're worst at simulating the behavior of other cars/pedestrians/cyclists/loose object.