You would need to stream all the sensor data in one direction while also streaming two 4k display streams in the other direction.
Working backward: frame needs to be received by headset in time to refresh, which means it needs to be sent by host device in time to make its way through the headset wireless phy and be decoded.
And working forward: headset needs to encode sensor values, send across wireless, be decoded on host, before frame can start to be generated.
Which leaves the time in-between those for actually doing all the processing and frame generation.
Ditching the wireless phy and low-level steps seems like an easy trade-off for more processing time.
Wouldn't that add more latency due to the extra encoding/decoding step?