> …it's a little bit refreshing to know that Apple recognized pass-through HMD AR as too hard and decided to invest in compensatory technology…
I understand the framing as "compensatory technology", but is it possible that what Apple's doing is the simpler and better way to solve the problem? Pass-through AR strikes me as an old-school analog approach, like optical printing for special effects. But a 100% digital vision pipeline seems like it could unlock interesting capabilities like "night vision", new ways of highlighting interesting objects, etc.
The other problem, of course, is that nothing can be truly solidly-colored. Everything has some opacity - which, combined with the FOV issue, is why HoloLens was never marketed as having anything to do with VR.
HOWEVER if someone can get the input -> photon production pipeline to be less than ~10ms, then that does solve a lot of the rendering issues, however it doesn't solve all of the other long term problems that come with that amount of hardware - including weight and complexity.
That said, there's a lot of known-unknowns that need to be solved, for example I don't have a solution for micropiezo resonance issues that I'm sure will crop up.
Currently, NVGs that are fielded by soldiers are already displaying images in a way that's not pass-through (using classic image intensification tubes). Something like the Apple vision headset in a lighter and more durable form factor would allow for eg; fusion imagery (fusing visible, thermal, and night vision).
$4k/unit is nothing when it comes to military hardware. There's also no need for the outward facing display, etc.
If anything, I wouldn't be surprised if Hololens goes in this direction. And the DoD already has contracts around the hololens.
- real NVG costs way above my budget, and all comes with defects. Realistically the tubes in civilian markets are all defective returns.
- even real NVGs can’t work in complete darkness, such as in a closet; moonlight or starlight is needed.
- real NVGs on firearms are often used in conjunction with an IR laser and a flashlight for aiming; advantage is solely that you have to have a goggle to see it, not that you can operate without making any emissions.
- partly due to above points, plain old CMOS cameras with modified 1” Sony sensors for better low light performance is starting to match practical usefulness of a real NVG(it’s Cold War tech so no wonder here). Latency is an issue, but more of “is an issue” and no longer a blocker.
… Vision Pro type of devices replacing NVGs indeed seem like a near future possibility.
They do work in complete darkness! You just need an IR illumination source. If not using a weapon mounted illuminator you can just use an IR flashlight. Thermal doesn't require external illumination but isn't anywhere near as useful for navigation. That said, good IR laser + illuminator weapon mounted systems are a few thousand bucks too.
My point about vision pro type devices is they can take a feed from real NVG tubes and potentially regular cameras and thermal and combine them. It'll be game changing when the tech is ready. It's not a matter of if but when.
I think they're different and not one better than the other. If I wanted to drive with an HMD on, or otherwise be in a situation where it could be deadly to have my sight turned off for even a second or to even have some lag or stutter or other glitch in my eyesight, I'd much rather have a pass-through AR HMD. One's sense of sight seems much more reliable with it by its very nature. You simply don't have those modes of failure with transparent plastic, no matter what's going on in the hardware/software.
Apple can make it possible if they wanted too.
Planes are crazy expensive anyway,so fly-by-wire can be afforded.
Also, it’s true that Airbus sidestick is fully electronic, but there’s a nuance in engine control being in the center on airplanes. Until automated, engine control on airliners was a task of a flight engineer, like it still is on ocean going ships. So fully digitalized engine control is a replacement to the FE, not necessarily an automation of what originally was a pilot’s task. Which means, I think, pushing a vehicle forward was never necessarily the responsibility of a pilot or a driver, though controlling where not to go is.
This is changing rapidly. eg:
https://www.infinitiusa.com/infiniti-news/technology/direct-...
Wearing the quest in pass though mode is kinda cool for a bit, but the lag makes it horrible to use for any more than checking where a wall is.
If they really have made this feel instant that’s a massive deal on its own.
But otherwise I agree, it makes sense for them to focus on what they can do best with current tech as a stopgap. With current see-through HMD tech, AR ends up incredibly disappointing. (See also: Hololens & Magic Leap's limited FOV)
With a projector, you can't "throw" nothing (aka black). As a result "projected black" is simply lack of projection.
In the case of a translucent or transparent reflection or waveguide surface - which is what the projection reflects off of - "black" is whatever the darkest part of the surface is. In effect whatever else is emitting from the surface that you're looking at will change the depth of "black" you get.
This is why the Hololens and other see through AR devices are always tinted, to set a higher threshold for "black" than the surrounding unaided view.
The are three layers of polarizing material. The two outer layers are at right-angle polarizations to each other and normally would be completely opaque on their own. When power is applied to the liquid crystal, it twists the crystal's polarization to be at a 45° angle to the other two layers, which then permits some of the incident light to pass through.
An optically transparent waveguide display can use an LCD layer to block light coming through the front and then not render graphics on that area of the display. It will be opaque black at that point (though rather fuzzy around the edges, as the LCD won't be in focus).
Magic Leap 2 actually employs this technique. It's... a lot like the rest of the device: a good idea on paper.
If you forgo the projector part and replace the mirror with a transparent LCD, it’s just too close to your eyes and you can’t see anything. If you add a microlens array to the LCD, now the LCD might come to focus but background becomes way too far-focused, and you can’t see anything either.
If we were on an Enterprise-D, I guess I could just ask replicator for a passive illuminated metamaterial light field image combiner with integrated processing than runs on bus power from DisplayPort input, but we are not there yet.
So, for now, our AR HMDs can only brighten pixels against backgrounds.
The blocking all light approach also allows you to hide other potential weaknesses of a device. For example, a lower field of view is much more distracting in a pass-through AR device as you still have your full peripheral vision. VR devices will generally black out the light outside of the FOV making it easier to ignore.
I mean... theoretically you can if you can match photon for photon with the inverse phase. Difficult, I know :)
1. Field of view is limited to existing optics miniaturization
2. Subtractive shading (rendering black) might not be solvable
3. Variable focus objects in the same scene requires projecting n>2 significantly different wavefronts - not solved how to do this with a single vibrating element
For example, close one eye and hold the tip of a pen about an inch in front the other and you'll see that it doesn't actually block any of the world.
Moore's law works great for semiconductors, but Maxwell doesn't negotiate ;)