I didn't read the paper, but am familiar with: https://mpeg-miv.org/

One challenge here is dedicated hardware support, running any sort of video processing/decoding on the CPU is unfeasible (GPU less so), particularly if the context is a battery-powered device (Quest 3, Apple Vision Pro), with high resolution and framerate (6K-8K, 60fps).

That doesn't even get into the logistics of capturing, serving, and software supporting this video data either