> Failure/slowness of any given application would leave the existing application geometry in-situ within the 3D environment and avoid the most jarring extremes of user experience.

This is already true with even the oldest and most rudimentary AR/VR systems, eg, Gear VR. The interface and head tracking is 90fps all the time. If the app is slow, the viewport will not freeze. Just it will be stuck on one frame.