Have you ever tried reading a book while being driven around? You get car sick really fast, because your vestibular system does not like constantly getting jostled around with mismatching visual inputs. Eating dinner while being driven around is a similar story: anyone around you does something unexpected, and your dinner ends up being spilled all over the car.
Self-driving cars will never have the same smoothness as planes or trains, simply because they have to share the roads with the general public. You might be able to sleep, but your commute isn't going to neatly match up with your sleeping schedule, so that's only really useful for longer road trips.
> this halfway autonomy where you have to watch everything is kind of useless
No, it is actively dangerous. Humans are extremely bad at staying fully-focused and being ready to act in the blink of an eye, while at the same time having literally nothing to do 99.99% of the time. If you don't let your attention wander, it is significantly less exhaustive to drive yourself. Halfway-autonomous cars exist primarily so that car manufacturers can demand extra money without having to accept any of the reliability.