I need to play with stuff with my hands (and I mean random stuff, not just always the same toy. Like right now I'm busying my hands with mounting/dismounting some lego that requires putting a bit of force in it, not really something you can do with a controller). I need to look at my smartphone. I might do interruptible stuff which would require corner vision (say I'm playing on TV, waiting for my build to finish that I can see in the corner of my eye). I'm chewing through gums (k I could probably do that with headset on)
Possibly all of those stuff could be fixed with better hw and sw. But. BUT. One of the feature I really like about current day VR, is that I am disconnected from everything. Which would pretty much contradict fixing every issue I have mentioned.