Meta's head of augmented reality software stepping down
reuters.com
reuters.com
For those paying attention, the technical VR "stuff that works" is basically written by Qualcomm.
The user side Android system is, really badly done and irritating for people using it as people like Carmack have pointed out.
This is possible the biggest software development blunder in history, they seem to have spent 10-20 billion dollars on... Qualcomm making a good vr headset for them and them failing to deliver a good experience with it.
I worked at Oculus on the product that became Quest from 2014-2019 and this statement could not be more wrong. You need more than a vulkan driver to make a VR headset, e.g:
* lenses that minimize distortion and chromatic aberration, that are lightweight and compact
* astounding amounts of work on inside-out tracking to make it work within latency and power budgets with sufficient accuracy
* latency reduction and prediction throughout the stack
* foveated rendering
* supporting all of those things in Unity and Unreal
> The user side Android system is, really badly done and irritating for people using it as people like Carmack have pointed out.
The user shell side we have today is entirely his creation, as a response to the less-than-stellar behavior of the Unity shell in the first release. I'm not sure Android is very material for the day-to-day user experience; it's just a particularly opinionated version of linux.
There are plenty of missteps here, but none of them are things you've described.
Its worth noting that much of that 10 Billion wasn't just spent on Horizon Worlds as critics off-handedly suggest, but has been sunk in to lots of acquisitions (hardware and software companies) and research, much of which has yet to be seen in released products.
I'm very impressed with the ambition of what they are trying to do and their committment to a longer term vision which is something that often is lacking in many companies. As an example Nvidia were working on AI tooling and hardware for a decade before it really started to pay off big.
Im not sure they will win this race, but the glasses are solid for the two instances i mentioned. Other features aren't as solid like Hey Meta barely works.
That was Corning, not Meta.
> astounding amounts of work on inside-out tracking to make it work within latency and power budgets with sufficient accuracy
That was Valve, not Meta.
> latency reduction and prediction throughout the stack
That was Carmack, so technically Meta, but it was also 6+ years ago.
> foveated rendering
That was a waste of time.
> supporting all of those things in Unity and Unreal
I'm sorry, I haven't seen the Unreal code, but based on what's in the Unity code, this being a "win" is laughable and maybe signals the low expectations you have to also think that Android hasn't had a material impact on day-to-day dev experience.
I don't know who manufactures them, but Meta has had a team of optics engineers working on new lenses for years.
> That was Carmack, so technically Meta, but it was also 6+ years ago.
Timewarp was one of many improvements, done by many different people over the years.
> That was Valve, not Meta.
Are you referring to the classic Valve room demo which used fiducials plastered on the wall for tracking, or the Valve lighthouse which projects lasers on the walls for tracking?
Whether you put fiducials on the wall and a camera on the headset, or vice-versa (like the original Rift did) is not that relevant of a distinction. Whatever "inside out tracking" meant in 2012, today it means you get 6dof without having to setup a pile of gear in your room.
> That was Valve, not Meta.
When did Valve make a headset with camera based tracking instead of setting up lighthouses all around the room? Their tracking that I'm aware of is all about the timing of light flashes sweeping across light sensors, it would be totally useless on Meta's self-contained headsets.
> > foveated rendering
> That was a waste of time.
If you're trying to sustain non-barf-inducing framerates on a mobile processor this seems useful to me?
But the alternative is the games wouldn't sustain acceptable framerates, so I'll take that tradeoff. I'm not saying it looks good, but it's running on a Snapdragon 835 so you do what you have to.
> That was Corning, not Meta.
You need more than a glass maker to build novel optical designs, besides which, none of the headsets at least within that time window used any material from Corning. The lenses in DK1 and DK2 were essentially straight copies of an off the shelf magnifier. The first consumer Rift used a Fresnel lens design that was re-optimized internally off of a design that started at Valve. The design was just the tip of the iceberg though. A colossal amount of engineering work went into manufacturability, especially around coatings. Rift S, Quest, and Quest 2 all used an evolution of that design that took in further advancements done internally within both the product and the research teams. One general note on all of this is that "lens design" for isn't a matter of drawing some curves and throwing it out to a factory.
Then you are required to install the oculus app on your phone and link it into your Facebook account. If you're lucky, this will work the second or third time you try it. Even then, sideloaded apps are hidden away in a separate menu.
Oh, and you think you can just use your quest tethered while you work in Unity? Good fucking luck. It's remarkably unreliable and unstable. Usually I get a black screen until I restart my headset, my computer, SteamVR, unity or a combination thereof. It's been over a year and I still don't know what the right incantation is. If you want to try air-link, you don't get hand tracking. Intentionally. It's not a technical limitation.
Beyond that, it's mostly-standard Android, which is also painful to develop for, but in new and exciting ways.
Conversely, targeting Valve Index and an ultraleap tracker is as easy as it gets. If your application runs on the PC where you're developing it, you're already done. It's plug and play almost every time. There's no extra layers of bullshit to wade through, you just open unity and hit play and that's it.
Well, unless you count all the ways that PC development sucks.
Development concerns aside, the quest is a low power phone with some cheap optics. Resolution is bad, frame rate is bad, battery is bad, it literally does not fit my face. I can't see anything because you only have three IPD choices and they're all wrong. Hand tracking is consistently worse than ultraleap, and with the latest update it's totally unusable for me. And I mean that I literally cannot select UI items. It nags me every 30 seconds about turning on my controllers. I also have this weird issue where different colors render at different depths. Red text floats above the surface and green is sunk below. It's really hideous.
The Quest is worse than a PC HMD in every way except that it's cheap. That's it, that's all it has going for it. It's cheap and technically does run VR.
I hate to be that guy, but please google "therac-25".
There's been absolutely no loss of life, just software shittily written. Yes, it's a blunder, but the hyperbole is unwarranted here.
Or "UK Post Office Fujitsu".
Wrongful debt, convictions, children removed from their parents homes, and even suicides.
The Post Office knew that their prosecution was unsafe but went ahead with it anyway - they had a duty to disclose to the defence any material in their possession that undermined the prosecution case or advanced the defence case.
This is a statutory duty and a police investigation is ongoing into the prosecution.
I think the shock factor in Therac-25 is the contrast between the gruesomeness of radiation poisoning and radiation burns and the post-mortem investigation showing the hubris and recklessness of the Software Design team, Risk Management, lack of HW interlocks etc.
MCAS deaths were precipited by greed which is bad but not shocking.
Therac-25 was an entire group of Design, Quality, Regulatory teams just literally YOLOing into patient deaths.
I would think Volkswagen's CARIOT is possibly a contender in that space too, soon.
Caveat: I know a bunch of people who are or were in various roles, vertically well enough spread across that org, to form this opinion over the last years.
It's an almost straight port of normal android into the headset which is at least familiar but they've really not made any progress in making it more VR and comfortable over time. Lots of small tweaking of whats probably not a great overall fit for VR.
For a direct example look at the room boundary system and setup.
Even with the new depth sensor and the color passthrough, it fails to figure out whether a table is a surface or a pillow of polygons, it fails to set reasonable wall boundaries (and then the ui is awful for addressing that) and then it nags and irritates people with those failures and misunderstandings.
Last I checked, the room software still only stores one room in your house. If you go to your kitchen for your coffee... remake the room boundary over again.
This is very fundamental for any AR to be built on and its terrible and unreliable.
Edit- also, last I checked, if you turn off boundary nagging with the devtools you cannot use the passthrough mode whatsoever.
Anyone saying that the Quest's software is obviously terrible lives on a different planet than me. The Quest is so simple and works so well, especially compared to other VR headsets+software I've used. I really like the Index too which I use a lot and think does some small things better, but I find it hard to deny the Quest is much more approachable and polished.
For a regular user, most everything seems to work well. Setup was easy, UI is polished and simple enough to get out of your way, and I personally haven't encountered any show-stopping bugs with it. Maybe I don't expect much out of it, as I see it more of an entry-point to actual VR applications I want to use, rather than something I'm more invested in.
If anything, I was surprised how well it worked for a product still so early in its lifecycle.
What constitutes accurate wall boundaries is a matter of opinion but it works well for me. I've never seen the nagging you're talking about, unless you mean when it loses tracking entirely?
Congratulations to Meta VR teams. You've created a system perceptually more transparent than a thin sheet of glass.
The UI is going to be the same that the other quest headsets have
I bought one to hold me over until the quest 3 came out but it had a wobbly lens flaw that made me nauseous. It turned out to be quite common. I had to return it. Luckily the quest 3 is now out.
In my experience the UI and everything in the Quest 2 and 3 works great so I don't understand where the original poster is coming from at all.
In my opinion, the technical know-how of Zuck was perfectly valid to build Facebook, but not this. VR/AR/anything with real-time world simulation and rendering is completely different from the web tech stack and could be considered to be a different world, with a different mindset.
Carmack obviously had the appropriate mindset, but AFAIK not the power to act. So Suck and his appointed staff probably took critical technical decisions regarding the software architecture of this project.
Given that we know that part of plan was to buy Unity back in 2015 and build their Metaverse on top of this engine, I seriously doubt this team really understood what they were doing.
Technical know-how is the least of their problems. The tech is irrelevant to me. Facebook's user-hostile and abusive actions have made it so that now anything they make is automatically something I want nothing to do with.
In my opinion, no matter how good the tech is, anyone foolish enough to participate in Facebook's "Metaverse" is just bending over, spreading, and asking to get screwed over and over again. The company has repeatedly demonstrated that it is toxic, criminal, and actively harmful to their users. No amount of fancy VR tech could ever be cool enough for me to let them use me. They can change their name as many times as they like, but they've already shown us what they are.
You got it! I have an original Oculus Rift (CV1) and when they were bought by Facebook it was like a kick to the gut. Felt as if Philip Morris or Monsanto bought the company that made my beloved headset. I wish they could be spun off or something so I could use the product with a good conscience. For a while, I could justify using it because my money went to Oculus, not Facebook, but now that they require a Meta account just to use it, no way Jose.
This guy was more on the tech side, but the fundamental problems with Zuckerberg's vision are not technical. This really sounds like scapegoating.
Along the lines of: "I was laid off from the last two places, and I will never need to work again".