The primary use case for passthrough is bringing stuff into VR, specifically your hands (when used in hand tracking mode), and your keyboard. Your monitor is projected into VR by being hooked to your headset. Any other function's utility has been traded off for these purposes.
For an example of this on Quest Pro, check out Wooorld. It's a fun hangout app and is arguably a much better experience in passthrough.
However, the Quest 2 and Pro apply some smart reprojection to correct this, and it feels like everything is almost exactly in the right place.
You do notice the artifacts of this reprojection when you’re holding objects close to your face, when you’re moving objects around, or when you’re looking at thin objects such as wires.
I also recall John Carmack talking about this topic in one of his unscripted talks at Meta Connect, but I’ll have to dig that up.
That's a non-issue. Lenovo Mirage Solo had a stereo cameras at a proper IPD and the immersion of pass-through on that thing is still by far the best thing I ever experienced in a VR headset. The little offset between the cameras and your eyes is unnoticeable unless you try to touch your nose or do other stuff really close to the headset. And if you really need to, you could get rid of that offset as well (45° mirror and mount the cameras vertical to have them capture the same image as your eye would).
The problem with most other VR headsets is that they either have only a single camera for pass-through or use the tracking cameras that are mounted far away from your eyes. Doing the remapping of perspective in software just doesn't give good results.
Seems obvious to me that the "Mixed reality" stuff is just marketing fluff, if you pay any attention to the technical evolution of this product line when interpreting these complaints.