We drive significant distances each year and I need to be comfortable when I drive. If the wheel alignment is out, it's an irritation having to continually push the wheel to centre and I get it fixed. If something in the load is continually rattling or squeaking (hate polystyrene) then I pack it so that it doesn't. Distractions like this are tiring and potentially create distractions.
I love the idea of fully autonomous driving where I can sit back and read a book or watch a movie, but I cannot imagine what it would be like driving a car where you are not actually driving, but have to be constantly on your guard in case it decides to do something unexpected.
Driving with constant attention is fairly easy, I would imagine not driving, but having to continually monitor ready to make split second would be exhausting.