VR optics and why IPD means too many things (2019) | Hacker News Reader

VR optics and why IPD means too many things (2019)

tomforsyth1000.github.io

121 pointsluu64 comments

When I built my VR MMO in 2014 I experimented with very wide camera separation, it's very particular; makes things look very small and distant but they pop out much more. Like 3D on steroids + tilt shift.

I also experimented with twice the rendering speed to get as recent as possible sensor reading before the Valve/Oculus warping became standard and it was VERY good for nausea. So 120/150 FPS respectively on DK1/2.

I'm looking forward to pancake lenses, if they remove godrays so I can play my game again: http://aeonalpha.com

But I read there are a lot of tradeoffs with them too.

So far DK1/2 lenses with cups that you could change and adapt to your diopter where the best optics VR has had (but I got lucky with IPD and one cup on DK1 being my exact diopter!)

I wish one actor made super tiny glass lenses just so we could see how that would work!?

Here is some more experiments: http://aeonalpha.com/fov.html

We had a somewhat undocumented 120Hz mode on DK2 where both eyes would get the same image (meaning everything was at infinity distance), but I’m not sure if any developers used it.

But then you get no depth perception?!

Yes, that’s why nobody used it. The plausible use case was something like watching 24hz, 30hz, or 60hz video content without juddering that 75Hz would have, and where depth didn’t matter.

Could you render very close objects twice, far objects once, then composite the image for each eye?

Theoretically, yes, but there will be a lot of overhead for Z-sorting objects as well as performing a multi-stage render like that. I am only guessing, I certainly have never tried, but I bet it comes out to a wash, at best. It'd be better to keep the rendering pipeline simple, if that were the case. Large-scale worlds will have LOD for models and terrain at distance, which will help with the rendering complexity of far objects.

You can also pre-process it and apply super-far details as a skybox image. A 2- or 3-block radius of real, 3D city buildings with a cityscape skybox looks pretty good. A forest scene can use object instancing to really clutter up the medium field and leave even a generic landscape skybox for great results.

It's funny. In my experience, stereo rendering is not a make-or-break issue in VR. I mean, it depends on the app design, of course, but a lot of apps can get away with monoscopic without most people noticing.

I had a regression in an app I was building where it was rendering in a monoscopic mode and it actually took me a long time to figure out what was "wrong" with the app, but apparently, I was the only one who thought it was wrong. It went for at least a month before I noticed a problem and another month before I eventually had the time to diagnose it. All during that time, nobody made any comment about it, people were enjoying the app.

I eventually figured out that you really only notice stereo rendering in modern VR headsets if you have a lot of near-field objects with which to interact and you interact with them with direct hand gestures (either hand tracking or controller bumping). So, you need stuff to be within a meter of the user for them to really notice "this isn't right".

There is a gap past a meter and before 10 meters where it's hit or miss if anyone is going to notice the lack of stereo rendering. The more the user is moving around, the less likely they are to notice.

And then, anything past 15 meters, I don't think there are any VR headsets on the market (other than maybe the Varjo VR-1, but I've never seen one myself) with fine enough pixel pitch that anybody is going to notice the lack of stereo rendering.

Now, that's just "where does stereo rendering take effect"? Part of what I think makes VR good is the ability to get up close and personal with the environment, to manipulate things in your hands, so typically I think "good" VR apps are going to need stereo rendering, but it was still surprising to see that it's not a universal issue.

I think 10 meters is also the point in real life where 3d stops being noticeable, too, so I'm not surprised your ended up with that number. (I just searched and found numbers between 18 ft and 20 meters.)

I don't think the pixel pitch is the limiting factor anyhow, because movement of the head means that sub-pixel accuracy is achieved by the brain anyhow. It's amazing what the human brain lies about in regards to vision.

Yeah, I haven't done the math, so I don't know what the exact boundary would be. Supposedly Quest 2's tracking is precise down to ~7mm, SteamVR to 75mm, so depending on the system, micro head movements may not be larger than IOD. Assuming we discount micro head movements, then the ICD is the only stereo cue. I think I remember seeing someone do the math once and said devices like the Oculus DK2 couldn't show anything beyond 10m for 0.07m displacement. Or something like that. Notionally, for some pixel size S, you will not be able to see a discernable difference at Z distance for X view displacement; the left and right eyes will resolve objects to the same pixels.

This is a "you may not be able to unsee" thing once you read this, but... one of the major problems I have with 3D movies is precisely that 3D based on image separation extends out way less than people think it does, because the brain naturally picks up with heuristics and prior knowledge and a lot of other things. So if you're on a closeup of someone's face or something, sure, it can be in 3D. But if you've got a big action scene with, say, helicopters flying around and shooting at people on the ground, the camera is likely to be farther than 20 meters away. If you can see any "3D" at all, it's wrong. It should all be at the plane-at-infinity for you. Which means that if you can see "3D", rather than being an epic fight to save the world or whatever, you've got a fight taking place on a diorama in front of you, about the size of a large table or something.

Even something just as the scale like the Transformers being about a block away from you brawling with each other should all be on the plane-at-infinity to you.

It kinda takes the drama out of 3D action. Technically, the 2D version of the movie is probably more actually accurate at the important times in a lot of genres.

Yeah, the vast majority of 3D movies have always had that diorama effect to me, though I tend to discount my personal experience with anything stereoscopic because I had lazy eye as a kid, ultimately corrected by surgery, but late enough that my brain mostly ignores the information from that eye. Not totally but mostly.

In any case, I don't find the effect unpleasant or pulling me out of immersion, because frankly I don't feel that immersed in AAA cg heavy action movies anyhow. To me it's like watching a fun demoscene with a neat effect.

The only time I recall it being a bit tiresome is when friends dragged me to a few horror movies where the 3d was obviously low effort crank it to 11 for this jump scare stuff.

Thanks for the warning, genuinely appreciated!! I actually opted not to read this comment, I enjoy the immersion.

How is it that there's any actual science left to be done here; when the "VR world" has been spending money in occasional waves since the 90s?

Shouldn't all this research have been done, and published, already? And re-verified, folded, spindled, mutilated, then recycled as firelighters. It's been 30 years and should mostly be adapting numbers optometrists know already.

The first big VR wave was so early it was almost pre-Internet, so I assume most research was done in corporate silos, like Disney. Those silos are black holes from which no information can escape, so everybody has to re-invent everything.

1991; I spent a week manning a booth at a construction trade show. We were beside the Autodesk booth, where they had a big show of "VR walk through of the Denver Airport". Several times every day, all week, people would stagger out of the demo and yak into the trash can provided for the purpose... which was just behind us.

I think that really showed me the potential of the VR business and the spirit with which it would be pursued.

1968 was significantly pre-internet [1], even beating out the first of ARPAnet by a few months.

[1] https://www.historyofinformation.com/detail.php?id=861

Since we have a lot of theoretical science, it's easy to imagine everything is solved in engineering. But when you dig into a new application you learn there are tons of gaps in knowledge. There is theory to draw on, but you need to combine and resynthesize to make it into something that works.

I have had experiences translating a professional model (similar to optometry prescriptions) and learn the model isn't described well enough to write in software.

Do you expect everything that existed in the 90s to be "complete" or just VR?

Good question. As a comparison we have been piling up dirt significantly longer than VR has been around and yet the physics of granular materials is still actively researched.

the software I was selling at the construction show in 1991 was essentially about piling up dirt.

The reason I was there to sell it was that my software had a round up / "fudge factor" applied to the models, based on real world experience; where as the competition consistently under estimated things because they used the nice clean textbook formulas.

Just because a machine has a 2.6 yd^2 capacity doesn't mean it will move that much dirt per load. And so on.

Applying back to the original discussion of VR; I expect there's been enough time and enough experience with the things like head mounted displays that there shouldn't need to be much original research into "how to do the optics" of same. I'm ignorant of optics generally (and much else) but there should be textbooks and people arguing over where those are wrong and such; by now. For this specialized purpose even. See the amount of literature on LCD displays by comparison; the basic optics there are known, right?

We did just have a major breakthrough in HMD optics not more than 10 years ago with "pancake optics" that have only started making it into devices you can actually buy within the last year or two.

Translation into products requires a study of humans that most companies are ill-equipped to apply. For example, the Oculus/Meta Quest 2 headset only has 3 "IPD" settings, and the face plate was designed to fit a somewhat particular skeletal structure.

Notably, while it fits comfortably on my Western friends' faces, none of my East Asian friends or family, including any women, found it bearable (too tight on forehead, tilted downward, light leaking from around nose) or were able to focus it cleanly and assumed "VR was not for them".

However, a Japanese headset, often reviewed in English as being too small or uncomfortable, fit me perfectly, and the optics were clearer despite having lower technical specs. How illuminating!

I do wish that companies that designed wearables would either disclose the dimensions of their testers or provide what dimensions matter. I'm larger than 50% of the people on the planet, and I fall within a standard deviation of the mean in the United States. However, no product designed by a US tech company has ever fit me.

The real answer is probably the same as the Alf Landon election prediction by Literary Digest in 1936. Because they only sent out ballots to addresses of registered car owners and people listed in the relatively new telephone directory, the poll results skewed wealthy, urban, and conservative - class bias in action.

Are tech companies in the US making the same mistake?

> Are tech companies in the US making the same mistake?

Yes, but slightly different, because they're missing less of their "addressable market" than the Landon prediction was. Their addressable market generally being people outside of Asia who have lots of disposable income. People in Asia tend to buy homegrown products which approach or exceed the quality, are easier to get, and cheaper. People with disposable income outside of Asia still trend toward the Westerner morphology. Still it's amazing with all the East Asians in big tech companies, that so many domestic products seem ill-designed to fit them.

Meanwhile tons of my furniture and other items that are designed and manufactured in Asia and then shipped to the USA seem to be shrinking and I don't fit in so many of them anymore. Not sure if that's just selling cheap things that have less material/shipping cost, or if it's matching the optimal sizes for East Asians.

> Meanwhile tons of my furniture and other items that are designed and manufactured in Asia and then shipped to the USA seem to be shrinking and I don't fit in so many of them anymore.

I based my recent purchase of a office chair on its copious complaints from very tall people after I found I couldn't sit back on a couch I ordered from Costco without my feet dangling in the air.

If we were neighbors, I'd offer to trade you!

VR AR researcher here.

It is scary once you see what the actual VR research landscape looks like.....

There is a clear disconnect between users of vr and vr researchers.

I don't know even where to begin. Too much incompetence. Too much disinterest. Too much bullshit just to get grant money. And nobody has any clear idea of what VRs application should be.

So far even for me there are very few applications where vr gives an actual advantage for anything.

Take it situational awareness, overview, ux, visual quality etc. Most of the vr apps I have seen have no reason to be in vr.

I hear ya! One of my hobbies is to be a somewhat known stage & exotic dancer / dance instructor in VR using 5-11pt tracking (depending on my mood and production quality). The number of "features" that get in the way, and lack of representation for VR dancers and performers has been frustrating, especially as the world moves more and more towards 3-point robot avatars.

Actually had a chance to provide my perspective to a FAANG Research Group, but I was laid off (and they were disbanded).

I can't see VR becoming useful at all for general productivity. AR? Maybe. It'd be cool to have more monitors without taking up space. But even in that use-case, the resolution of current AR/VR tech isn't enough.

Even if you did find a way to make VR work for you at work, current headsets are just too heavy to want to wear them 8 hours/day.

They're still incredible for gaming, though.

Replace "VR with "cars", and say all this in 1923. Same thing. There's a lot to be done, few domains of knowledge are completely mined in 30 years.

1923 cars had about the same wheelbase, and steering gear variety, and differentials... the basics were all worked out and agreed on.

I really like the suggestion of looking at the HMD’s faceplate and having it measure your IPD. I think that’s a great idea!

As they say, it’s hard to adjust it optically by moving the lenses because there’s a wide range of values that give a focused image - but the wrong values will result in nausea over time.

I recently went from a quest 2 (limited 3 IPD settings) to a pico 4 (that has gradual fine adjustment) and makes a huge difference. Specially in the headache and nausea department.

Even with pancake optic artifacts, just that setting alone makes a huge difference in don't feeling dizzy in rapid movements.

The varjo xr headsets use the eye trackers to auto adjust the IPD mechanically the moment you put the headset on.

The problem is it takes a second or two and it fails.... Often, so you need to repeat the (take on and off). It makes development and multi person demos into a pain of "put it in and off again it will work"

The Quest Pro does the same, with the fit adjustment app.

Yes I agree something like this is probably the future, once it’s more reliable and more affordable.

For now only expensive headsets have the hardware for this. The solution proposed in the article works with the hardware in more affordable headsets which makes it intriguing.

Really great article. I’ve never heard before that VR headsets produce collimated-light images, but the explanation makes a ton of sense.

I found the explanation lacking. It just states that collimation is the case without explaining why. The why is the limited optics of HMDs. In the physical world that those optics are trying to simulate, photons can leave from any point in space at any angle.

Is there a graphical explanation of the perceptual phenomenon the author is describing? I think I understand the physical stimulus he describes, based on diagrams in papers, but I havent found anything explaining this part:

> So translating the pupil around doesn't change the angle of the light rays, which means the image doesn't move.

Think of the sun. Sun rays are collimated, yet when you look up the sun doesn’t fill the whole sky.

That’s because where in your visual field a light will end up depends on its angle, not on where it hits the outer surface of your eye.

> photons can leave from any point in space at any angle

Not really. They can leave from any point on a surface at any angle. Once they're on their way, they don't change direction. If a surface is an infinite distance away, then by the time they reach your face, the photons from that surface are all travelling parallel to each other. That's what collimation is simulating: looking at objects an infinite distance away.

Why? As the article says:

"But try this experiment. If you can find a fixed test pattern, put that on the HMD's screen, then pull the HMD away from your face a little, and move it side to side by an inch or so (don't rotate it, just translate it). The image stays fixed in space! It’s a really strange thing – you’re moving the physical box, but the image inside it doesn’t move. That’s the power of collimated-light images, and it's still very unintuitive to me even though I understand the physics and maths very well."

Just how, when you look at a surface a great distance away, its appearance doesn't change if you move your head a few millimetres from side to side.

"Of course real VR lenses are not perfectly collimated for various reasons. But they're close enough that getting the IAD (optical axis separation) adjustment perfectly right is not as important as it appears at first."

Collimation means that the device is robust to the exact distance between the user's eyes. That is very useful.

>from any point on a surface at any angle

Yes, obviously what I mean. They don't just pop into existence - at least not for these purposes. But those surfaces can be at any point in 3d space.

Wow, what a fun and knowledge-filled article. It felt like reading Feynman.

Just casually browsing HN, this solves a problem that's been bugging me for months in my game: The world is scaled in feet/inches, while WebXR uses meters/cm to set ICD (virtual camera distance)! I think- based on my getting unusually nauseous.

I thought the solution was to scale the world, but there's more to it - and I think the solution is to reduce camera translation motion by the scale difference:

> Specifically if you translate your head sideways by the distance of your IOD, your left eyeball should now be seeing what your right eyeball was seeing. If it doesn't, the brain worries that something bad has happened, and makes you feel bad about it.

> Note that if you scale BOTH – you move the virtual cameras closer together, AND you reduce the in-game motion of the cameras due to head motion – everything is totally fine! The world just grew or shrank magically, but it all still works and you don't get nausea.

You may wonder why in hells a programmer would use imperial units instead of metric. It's to give the game a sense of being human-scale and pre-civilization. Why not just use metric and then convert everything to imperial for display purposes? Because then during debugging, units in logs constantly have to be converted to units shown, and the conversion isn't identical. It never bit me until VR!

Imperial units are not “human scale” any more, or less, than metric: https://xkcd.com/526/

A 1 foot sandwich can be easily shared with 2, 3, 4 or 6 friends. And nobody has 5 friends anyway. There are no meter long sandwiches .. and if there were, you couldn't split it between 3 people. It's impractical. It's like if there were 100 degrees in a a circle.. You'd go nuts

Metric is for when you've made a mess of things and have completely given up on having nice round integer values

EDIT: This comment has been timed to coincide with when Americans wake up to defend me :) ~ USA! USA!

Splitting a meter-long sandwich between three people is easy. You give each person a footlong slice, and the remaining 8.6 cm to the dog.

Or, you know... You look at it, more or less find the one third point from the tip, cut it there, then cut the remaining larger part in half.

Which is probably the same way you'd do it if it was a 3.28ft-long sandwich instead.

Impossible. Ever since Hercules accidentally stepped on his lunch sandwich in the stables of Augeas, his 12-inch foot has been the standard measurement for Ideal Sandwich.

In The Netherlands you can order beer in meters.

At any spring break resort or in Vegas, you can order beer by the yard

Edit: mustn't forget the French Quarter either

Obviously a meter of beer is better than a yard of beer.

Those last 3" (76.2mm) would just be foam anyways

Foam is measured in fingers.

Depends on how thick it is, doesn’t it? Why the insistence on measuring the length? Couldn’t you just as easily talk about 1kg sandwiches?

I dunno, a quarter meter sandwich seems just as natural as a quarter pound burger.

> It's like if there were 100 degrees in a a circle

There are 100 gradians in 90 degrees, does that count? They're probably supported on your calculator. https://en.wikipedia.org/wiki/Gradian

It's a similar kind of thing to decimal time: https://en.wikipedia.org/wiki/Decimal_time

I could down a meter long sub by myself, and I’m fairly certain my dog could too. No need to share.

I never understood why we never popularized a comparable metric measurement. A quarter-meter perhaps?

A decimeter is about 5 inches, which is the height of a cup, diameter of a burger, span across a hand, etc. To me that's a lot more human-friendly than a 'foot' being 12 inches.

“Hand” is already a historical unit, equalling almost exactly 10cm; i.e. 1dm. Horses are measured in hands.

<https://en.wikipedia.org/wiki/Hand_(unit)>

You are citing a cartoon with a dick joke and a mom joke to say that metric is just as human oriented as imperial. Seriously: Imperial: 1 Foot ~ the length of some guy's foot. 1 Second - the duration of one heart beat. Metric: 1 Meter - some 1/10000 of 1/4 of an mis-measurement of the circumference of the Earth. 1 Second - stolen from the Imperial system cause 86 microdays did not easily divide by 10, 7, 24, 60, 30, 31 or 29.

The first mechanical clock that could accurately and reliably measure seconds was a pendulum clock with a length of one metre. Seconds and metres go hand in hand with the metric system.

I mean, sorta? First, arbitrary numbers are arbitrary. But I can point to my foot and say that is about a foot in size. An inch is roughly one of my knuckles. A yard is roughly a step.

Even that comic you linked drives this home. None of them are referenced to humans. They are things we mostly know, sure. But they aren't based on a human. (Well, I suppose the mass ones have an amusing set of people listed.)

And this avoids the measures that, for my memory, just haven't made it to metric yet. Acre, Astrological Unit, Knot, etc.

"3 L - TWO-LITER BOTTLE"

Um, okay...

As computer specialists, we should advocate a return of the Imperial system using fractions of an inch for measurements. This is because it fits perfectly into the binary floating-point system. There are no more problems relating to 0.1 being an irrational number in binary. It's all broken down into 1/2, 1/4, 1/8, 1/16 etc which in binary is 0b0.1, 0b0.01, 0b0.001, 0b0.0001

Reply on news.ycombinator.com