I'm waiting for one of the headset vendors to support IMU-based HID mouse cursor control in hardware, with the same end goal in mind. In the meantime I'm stuck on amd64 with my little libinput driver[0].
To answer your question directly, I've done multiple 2-3 hour sessions at a time without taking the glasses off and have done 10+ hour days in them when away from home. At home I typically tend to use them for a couple of hours in the afternoons when I'm tired and more willing to sacrifice screen real estate for comfort.
The detail isn't the same and you have to plan your screen layout a bit (i.e. looking at code near the edges of the screen is annoying). But I think they're the future - a much bigger deal than VR or AR.