In every similar story I always fail to imagine how the driver can possibly stay in a state of "active attention" 100% of the time, while also not doing anything active. That's not how much of the human race works.
At this point it's a lot easier to just actively drive, instead of having to make a sudden split-second decision while being dulled by the inaction. I feel like non full-autonomous driving just doesn't make sense at all.