Comment by tokyodude | Hacker News Reader

Comment by tokyodude | Hacker News Reader

Parent Full thread

But you don't really. (Well, I don't).

It seems obvious to me that there's a progression. Back in Apple II, C64, DOS days computers generally ran a single app at a time.

Windowing systems were invented allowing multiple apps to share the same screen. This is what we've had for 30-40 years.

For AR though we need multiple apps to share the same 3D space all running at the same time just like I have 14 apps running on my laptop right now.

I need my mapping app drawing the path to my destination. I need my transportation app showing me the next bus/train coming. I need my virtual pet app adding my virtual pet around my feet. I need my social networking app to show me my friends info next to them in me view. I need my dating app highlighting people that are potential matches.

All of this has to happen simultaneously and at 90fps. That will require an entirely new OS and possibly new hardware (dual GPUs, ... or pre-emptable GPUs) one for compositing the 3D world at 90fps and one for apps so if app takes too long to generate it's portion of the scene the entire OS doesn't start running low. An old OS that only runs a single 3D app at a time and who's apps don't share the same 3D space will be a huge hindrance to AR being truly revolutionary.

I'm not saying Magic Leap's OS is that OS but that OS is not MacOS/iOS/Windows/Linux/Android

> I'm not saying Magic Leap's OS is that OS but that OS is not MacOS/iOS/Windows/Linux/Android

One of those is not like the others: “Linux” is not a graphical environment. If you had said “X11” or “Wayland” (or even “Gnome” or “KDE”, etc.), I would have agreed with you, but “Linux” could perfectly well be the base of an entirely new 3D-based application environment.

> For AR though we need multiple apps to share the same 3D space all running at the same time

This is something that I've given some practical thought to. The approach that I considered was to treat each application process as a client to a '3D space' service through which it could add and modify defined geometry.

Failure/slowness of any given application would leave the existing application geometry in-situ within the 3D environment and avoid the most jarring extremes of user experience. I imagined that the service interface itself would mandate the availability of simplified geometries together with meta information. This would allow the service to appropriately degrade the 3D rendered environment to maintain high framerates.

I think this approach could be effective and would not require a new OS.

I had a similar thought a few weeks ago, but for smartphone camera and mapping apps (and the intersection of the two when it comes to world-based AR). Rather than every new app out there having to implement their own camera and map view, have them provide an extension that can be called by the native camera/map apps (providing the view bounds/coordinates/etc), that returns a rendered layer that gets overlaid on the view.

Examples:

In my state we have a govt funded web/native app that provides the fuel prices at all the service stations around you. If there was a mapping extension you could turn on, it would show this directly within the mapping app, and also an indicator "cheapest fuel within X km is at Y location" when you're in navigation mode.

In camera view with an extension turned on, provide object detection similar to how QR codes are recognised.

SteamVR basically does this - it is a very evolved implementation of a VR OS.

Apps which crash or otherwise stop rendering trigger a fade out to the white-room VR space, whereas frame-skips are handled by hard drops (so the effect is like a teleport to avoid motion sickness).

Like, Valve have really done an incredible job on the user application interface there.

> Failure/slowness of any given application would leave the existing application geometry in-situ within the 3D environment and avoid the most jarring extremes of user experience.

This is already true with even the oldest and most rudimentary AR/VR systems, eg, Gear VR. The interface and head tracking is 90fps all the time. If the app is slow, the viewport will not freeze. Just it will be stuck on one frame.

Why can't that OS be one of the standard ones?

Sure, it might require special hardware and software (even your screen has it's own software and processing that it is doing on the signals coming from your desktop), and at the end of the day the OS would need to change and evolve to handle this new paradigm, but it's not impossible to imagine a system where the "PC" does what it does now, and just tells the "VR" sub-processor/sub-machine where to put the rendered "window".

Already there are some VR apps that basically wrap your normal "windows" desktop in almost a cylinder around you and you can move screens around on it as much as you want with current VR headsets.

Imo it's not that the traditional OSs can't be the OS of the future. It's that the future OS needs to be developed from scratch without carrying over the assumptions and obligations to older design principles. Historically, this has been very difficult. E. G. Cocoatouch (ios) has significant differences to its cocoa (osx). Bottom line is you have to go all to do it right, but its risky for those big companies that have a financial interest in squeezing out as much value as possible from their cashcow OSs

One of Andromeda's target platforms is AR (and not just HL2.) And Linux of course gets it. Android, too. Plenty of experimental builds out there, I'm not worried about pivoting.

And those are terrible experiences.

Reply on news.ycombinator.com