How does the author justify comments about 16K resolution being optimal, 12 cameras being the minimum required, and 240Hz being ideal?
That leads to several hard problems: you don’t just need to generate a high-quality scene, you need to render it twice because your brain is used to the distance between your eyes (16K==8K per eye) and because your position isn’t static it needs to update very quickly when you move because your brain is very used to how that works in real life. You have a budget measured in single digit milliseconds to that or a fair number of people will get motion sickness and many more will notice the lag and find it detracts from their experience.
Here’s a good overview:
https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
That's not how Ks work.
And your explanation doesn't cover why the minimum number of cameras would be 12.
> And your explanation doesn't cover why the minimum number of cameras would be 12.
This is covered in the article. The key thing to understand is that there are multiple tasks using cameras: AR needs to track the world around you, you need eye tracking to handle saccades and conscious direction, and tracking hands, arms, legs, facial expressions, and body position accurately requires multiple cameras with enough parallax to build 3-D models which are accurate enough to generate realistic, stable imagery. People will really notice if things bounce around or deform or if you punt on some of the proven and end up with Meta’s leg problem.
The key point to take away here is that AR/VR aren’t one hard problem but a collection of hard problems which all need to be solved for most people to find the experience compelling — all with tight physical constraints.
Nothing explains why 12 specifically. And if 12 is a minimum, what's the optimum?
> The key point to take away here is that AR/VR aren’t one hard problem but a collection of hard problems which all need to be solved for most people to find the experience compelling — all with tight physical constraints.
And it's a pretty bad point because it only works if you constrain yourself to head-mounted (including battery) standalone devices. The article gives a nod to other ways of doing it, but otherwise ignores them for all its main arguments.
And you don't need optimal to be compelling.
The article isn’t a complete survey of the entire field, it’s talking specifically about the most popular approaches and extrapolating what the author believes a breakthrough popularity device will look like. It does mention other options such as Apple’s rumored support devices or how industrial applications can take advantage of their environment. The challenge here is price: these systems are expensive and getting people to buy them means they either need to be cheap (impossible) or give a lot of value, and companies are assuming that means something you can easily move around in and use for long periods of time (not many people are going to set up a VR room, buy a headset and sensor array, and a pricey PC).
While it’s true that it doesn’t need to be perfect, some of the hard problems spoil the illusion - people get sick or suspension of disbelief fails - and it has to be enough better than the widespread mainstream entertainment systems to justify the much greater cost. Currently we haven’t hit the point where many people, even many self-described fans, use these systems heavily and that makes it an expensive hobby.
How many minutes per day would you consider “heavy usage” given the current entertainment/gaming focus?
This ties back to the price question: this currently costs noticeably more than console or even PC gaming, both of which are more social. A mainstream product needs to offer something you don’t get there to justify the extra money.
But I think you’ll find that your priors are quite a bit off at least in terms of “heavy usage” of VR though. Without any snark, I highly suggest talking to people who work in the industry if you’re curious.
Note that a new Quest 2 costs $400 with the price hike (previously $300, and sometimes as low as $250 when on sale). Compare that to $500 for a PS5 or Xbox Series X. I don’t think it’s correct to say that VR “costs noticeably more” or doesn’t have something extra to offer; don’t discount the novelty factor of VR to move sales.
The Quest 2 is reasonably priced, yes, but every review mentioned drawbacks (often due to Facebook rather than the concept) so the question of satisfaction per dollar is still pertinent. The people I know who are VR fans all use PCs which is where my cost priors come from - high-end gaming system rather than console level - but hopefully that’s shifting. I was definitely expecting this to be more noticeably popular by 2023.
Assuming inside out tracking, redundancy is required, since you can make the majority of the cameras useless by standing in the corner of a room with white walls. Features are required for tracking.
Maybe the future is LiDAR, or something new, or old (military gets along with IMUs).
Well, we're talking about the future. The Quest 2 implementation proved that the future is definitely not lighthouses [1], especially for mobile, which is the neccesary future of AR. Lighthouses are a technology from a time when there were no SOCs with the hardware required to support the cameras. In the end, we'll probably end up with a mix of SLAM cameras, and lidar.
Full body tracking is all that's keeping lighthouses relevant. Maybe that goes away with more cameras. :) If you have enough cameras/lidar, you get tracking of all world objects, not just those with sensors attached.
1. https://dl.acm.org/doi/fullHtml/10.1145/3463914.3463921
> The results reveal a significantly higher accuracy for the Oculus Quest 2 compared to SteamVR Tracking in the height of a tracked object. Furthermore, the Oculus Quest 2 tracks its position with substantially higher precision than SteamVR Tracking.
Only if they're cheap enough, but if you're in the situation of "cameras are so cheap we'll put in 40" then the idea of "minimum 12 cameras" is extremely not an issue.