Namely, you can't be certain you know where the "camera" is.
Aren't you assuming that they would allow the wearer to change the camera position?
You could film a movie or design a game such that the camera location is fixed or on rails. You're more like an observer than a participant.
Eg. Chronos and Edge of Nowhere