" Obviously more information is needed, but I thought the entire point of having a driver behind the wheel is to manually intervene to prevent this very situation?"

I think it's time to point out the obvious, and require that autonomous cars apply the brakes first, and THEN require driver intervention.

And that they be a whole lot quicker to err on the side of braking.

Cameras getting fuzzy ?

Slow down.

Your ML algorithms are showing lower confidence measures for how they classify nearby objects and trajectories ?

Slow down.

nearby vehicles slowing down and you don't know why?

Slow down.

This is inexcusable.