Woe to you if Debian pushes a systemd update. It took repeated incantations with apt to get that update to take, because updating systemd would crash the VM Every. Damn. Time.
[0] The current console-only incarnation.
I'm cautiously optimistic about AVF, but I don't see why it would have any better performance than native code running directly on the host system?
To answer your question directly, I've done multiple 2-3 hour sessions at a time without taking the glasses off and have done 10+ hour days in them when away from home. At home I typically tend to use them for a couple of hours in the afternoons when I'm tired and more willing to sacrifice screen real estate for comfort.
The detail isn't the same and you have to plan your screen layout a bit (i.e. looking at code near the edges of the screen is annoying). But I think they're the future - a much bigger deal than VR or AR.