Hacker News Reader

Comment by CharlesW | Hacker News Reader

(Caveat: I don't know a lot about pass-through HMD AR, and I assume you're incredibly smart and that the patent is innovative.)

> …it's a little bit refreshing to know that Apple recognized pass-through HMD AR as too hard and decided to invest in compensatory technology…

I understand the framing as "compensatory technology", but is it possible that what Apple's doing is the simpler and better way to solve the problem? Pass-through AR strikes me as an old-school analog approach, like optical printing for special effects. But a 100% digital vision pipeline seems like it could unlock interesting capabilities like "night vision", new ways of highlighting interesting objects, etc.

View on HN

I think the most prominent attempt at see-through AR was the Microsoft HoloLens. But if you've actually tried the HoloLens, the Field of View is atrociously, tragically small. The first HoloLens had a field of view of 30°*17.5°. The second HoloLens improved to 43°*29°, but it's still best described as "cramped." Couple that with almost all of the compute budget for the device going into vision processing and having very little compute left for actually running apps (the first HoloLens having a 1Ghz Intel Atom from 2015, the second a superior... Snapdragon 850).

The other problem, of course, is that nothing can be truly solidly-colored. Everything has some opacity - which, combined with the FOV issue, is why HoloLens was never marketed as having anything to do with VR.

The vision processing was done with a separate SoC. Your applications were not contending for compute time with the vision processing. That's just how anemic the CPU was.

> is it possible that what Apple's doing is the simpler and better way to solve the problem? Pass-through AR strikes me as an old-school analog approach, like optical printing for special effects.

I think they're different and not one better than the other. If I wanted to drive with an HMD on, or otherwise be in a situation where it could be deadly to have my sight turned off for even a second or to even have some lag or stutter or other glitch in my eyesight, I'd much rather have a pass-through AR HMD. One's sense of sight seems much more reliable with it by its very nature. You simply don't have those modes of failure with transparent plastic, no matter what's going on in the hardware/software.

Your steering wheel and pedals are digital too. Airliner controls have been digital for decades. Factories have life or death safety systems that count on silicon and a software stack to respond quickly and without error.

Apple can make it possible if they wanted too.

These are all very different from the “HUD goes dark and my meat brain panics, causing me to get into an accident” scenario that GP is talking about.

There are multiple system running on silicon that have been made to not go dark in normal mode of operation. Some examples could be found at https://en.wikipedia.org/wiki/Safety-critical_system

I believe cars' steering are analog because it would too expensive to achieve the required level of reliability.

Planes are crazy expensive anyway,so fly-by-wire can be afforded.

Steering is analog. One of reasons why by-wire steering control did not succeed is user feedback, it was tried and failed. Brake pedal is analog too, just assisted and override-able. Gas pedal had been all digital for some time.

Also, it’s true that Airbus sidestick is fully electronic, but there’s a nuance in engine control being in the center on airplanes. Until automated, engine control on airliners was a task of a flight engineer, like it still is on ocean going ships. So fully digitalized engine control is a replacement to the FE, not necessarily an automation of what originally was a pilot’s task. Which means, I think, pushing a vehicle forward was never necessarily the responsibility of a pilot or a driver, though controlling where not to go is.

>Steering is analog.

This is changing rapidly. eg:

https://www.infinitiusa.com/infiniti-news/technology/direct-...

That was the failed attempt I was referring to, literally a decade ago. Toyota is doing a retry with bZ4X/Lexus RZ, and those three are the only known cars to have passed prototype stages with (front wheel)Steer-By-Wire.

Which is probably the point of the R1 and the real-time subsystem.

Yeah this is the most interesting bit in a way.

Wearing the quest in pass though mode is kinda cool for a bit, but the lag makes it horrible to use for any more than checking where a wall is.

If they really have made this feel instant that’s a massive deal on its own.

If 1T Apple are such magic tech wizards then why couldn't they make AirPower work?

In theory the wavefront emulator is the simplest display - all of the "pixels" are rendered in your brain so you don't have to build an actual "display"

HOWEVER if someone can get the input -> photon production pipeline to be less than ~10ms, then that does solve a lot of the rendering issues, however it doesn't solve all of the other long term problems that come with that amount of hardware - including weight and complexity.

That said, there's a lot of known-unknowns that need to be solved, for example I don't have a solution for micropiezo resonance issues that I'm sure will crop up.

Agreed. In fact, this approach is also likely better for military applications which I hope they explore.

Currently, NVGs that are fielded by soldiers are already displaying images in a way that's not pass-through (using classic image intensification tubes). Something like the Apple vision headset in a lighter and more durable form factor would allow for eg; fusion imagery (fusing visible, thermal, and night vision).

It's not in Apple's interests to get into military contracting because of the potential to damage their brand, as well as avoiding the geopolitical tensions that would invite.

Apple won't. But as soon as these headsets hit the market manufacturers will be tearing them apart and reverse engineering them.

$4k/unit is nothing when it comes to military hardware. There's also no need for the outward facing display, etc.

If anything, I wouldn't be surprised if Hololens goes in this direction. And the DoD already has contracts around the hololens.

I’ve always wanted an NVG to play with, and from watching firearms nerd rants I’ve learned:

- real NVG costs way above my budget, and all comes with defects. Realistically the tubes in civilian markets are all defective returns.

- even real NVGs can’t work in complete darkness, such as in a closet; moonlight or starlight is needed.

- real NVGs on firearms are often used in conjunction with an IR laser and a flashlight for aiming; advantage is solely that you have to have a goggle to see it, not that you can operate without making any emissions.

- partly due to above points, plain old CMOS cameras with modified 1” Sony sensors for better low light performance is starting to match practical usefulness of a real NVG(it’s Cold War tech so no wonder here). Latency is an issue, but more of “is an issue” and no longer a blocker.

… Vision Pro type of devices replacing NVGs indeed seem like a near future possibility.

I have a few NVGs. They're definitely expensive (a good setup is $10k+) but it's definitely not true that they're all defect returns. L3 image intensifier tubes that enter the civilian market are just as good as the military gets for the same product. For cheap (<$10k) NVGs that's a totally different story though.

They do work in complete darkness! You just need an IR illumination source. If not using a weapon mounted illuminator you can just use an IR flashlight. Thermal doesn't require external illumination but isn't anywhere near as useful for navigation. That said, good IR laser + illuminator weapon mounted systems are a few thousand bucks too.

My point about vision pro type devices is they can take a feed from real NVG tubes and potentially regular cameras and thermal and combine them. It'll be game changing when the tech is ready. It's not a matter of if but when.

Reply on news.ycombinator.com