Mozilla’s Servo Team Joining Mixed Reality
blog.servo.org
blog.servo.org
Every time I see phrases like this I just cringe. As someone who spent a ton of time in the gamedev/3D UI space I've yet to see anything about VR that points to be being successful in anything other than small markets. I think there are some amazing applications in architecture and gaming but I really don't see something you have to strap to you face making the mainstream in a meaningful way.
I really hope this doesn't dilute Servo's effort too much because they've been doing awesome stuff lately.
I'm constantly surprised how blown away people are when they try (real) VR for the first time. That's what makes me optimistic about it as a medium.
(of course a badly set up machine can cause problems. If the frame rate tanks or the tracking is messed up then all bets are off)
We had few "VR experience center"s in our little city, they were popular when they just opened, but now most of it are closed.
So, I think "blown away" is not enough, usefulness is important.
Maybe the barrier to entry for VR hardware is low enough now that some small team will invent something that makes it a must-have, but I remain as skeptical now as I was when Oculus first hit the hype train. I haven't really used an Oculus or a Vive, so I hope this isn't coming from a place of abundant ignorance, but VR just feels like hype feeding hype. I haven't heard a VR concept that justifies it yet. The only one I even want to try is Tilt Brush.
There's only one way to find out. I'm over a year into VR ownership and it still fascinates me.
I'd prefer to use words along the lines of "inspring", "enchanting", "compelling" if we're going to discuss the potential of VR.
(I'm not belittling the large number of practical applications of VR in healthcare, training, education and content creation - but I don't think they will drive mainstream adoption in the way that creative, immersive applications could)
Watch out when comparing current typical art mediums with VR, a technology still in the property of a 2-3 big tech companies, paired with countless patents, and which costs hundreds of dollars, not including the top gaming rig that you also need to have.
Currently, if you want to convey a message, or exprime yourself creatively: - via music, you can buy any kind of instrument off the shelf. - via movies, you can just start with a $200 camcorder and record stuff. In both cases, consuption can be done with any device compatible with the digital format.
right now it's not possible with VR, at the state of things.
VR = an entirely new medium.
Just look at the number of arts projects that use VR. I hear about something new weekly. I don't recall artists around the world getting excited about 3D movies.
(Yes - VR has some similarity with games and other interactive content but cinema has some similarity with theatre and the novel has some similarity with poetry)
I happen to think that VR will not disappear as it clearly already is useful for certain applications. It might even be widely-used by general public some day. I am reasonably sure that day will not be in next 3-5 years.
I do however find comparisons to 3D TV/movies glib and unenlightening.
I agree that VR is a different medium and that 3D movies probably aren't, but you lost me at why that should matter. I still don't understand that, but that's fine as I also lost interest when you made this personal. I will not respond to any further comments in this thread.
Btw, I think we have common friends in Brighton. Your name occasionally came up when I used to work at Aptivate.
Hey! It's so odd when the virtual and real overlap. :-) I'll be seeing Alan this week hopefully.
Absolutely not something that mozilla should be focussing on though.
Servo has produced lots of great browser tech. We're seeing that tech make its way into Firefox with Stylo and the Quantum release, and there's on-going work to bring even more tech into Firefox, like the WebRender work that you can try out in the latest nighlies. Servo's goal is to create the best technology for working with both current and upcoming web standards, which means collaborating with multiple product teams, from Firefox to Mixed Reality to other, future explorations.
There’s still a long road to full compatibility with the existing web in all of Servo’s components, and it will take time to get there. In the meantime, though, there are emerging technologies where Mozilla believes it is vital for the open web to play a central role. One of these is Mixed Reality (which refers to both Virtual and Augmented Reality), a space that’s getting a ton of attention from all of today’s tech giants.
Mixed Reality is interesting for Servo in two different respects. First, it’s a huge opportunity: it’s early days and content is brand new, so there’s no long tail of web compat to worry about; we can get products built on Servo to market relatively quickly. By putting open web tech on the cutting edge, Mozilla can help ensure that Mixed Reality doesn’t become yet another siloed technology.
But second, the constraints of Mixed Reality will help us push Servo technology to the limit: we need to achieve 75 or 90 frames per second per eye to make a workable product. The research advances here will pay huge dividends back in traditional browser engines like Firefox.
In short, this is a “yes, and” shift. Servo continues to be about building the best browser tech, period, for use across Mozilla’s products. The increased emphasis on Mixed Reality represents an opportunity to push that tech further, sooner. And the organizational change within Mozilla Research reflects a closer collaboration between the teams needed to make that happen.
Web browsers are still written in C++. It is still bad for humanity.
Emphasis on mixred reality may or may not solve that problem sooner. Sometimes the quickest way can seem roundabout. But I am skeptical.
What I'm reading from the top comment here is that the actual point is to focus Servo on VR and Mixed Reality, because it will pull Servo even more in the direction it was always supposed to go in. That is to create a browser that outperforms everything else by leaps and bound by being able to more easily take advantage of multicore processors. VR and Mixed Reality require 90 fps in order to work properly for users, so this is a very high and a completely hard baseline for Servo to have to hit. So it's about setting a loftier goal and a higher bar to meet in order to make sure Servo reaches it's original goal of great performance by way of parallelism.
The problem for humanity is not that browsers are written in C++, it's that browsers have lots of security vulnerabilities.
Being written in C++ is not helpful with that, but it's not integral to the problem. It's not impossible to produce C++-code that doesn't have vulnerabilities, it just requires a lot of effort and often years of battle-testing to close all of them.
But Firefox's source code has for the most part had those years of battle-testing. It's probably safer than if you'd completely rewrite it in Rust, at least in the short term.
Where the use of Rust can deflect most vulnerabilities is in new code. And that's what Mixed Reality is. It's gonna need to be in the browser at some point in the near future and it is a big chunk of new code. It also has harsh performance requirements, meaning they'll have to work with parallelism, which is where C++ is particularly error prone.
Firefox is not safe. It’s been routinely exploited by law enforcement and hackers alike.
But there is no choice. Rewriting Firefox from scratch is going to take decades. Firefox has to continue to function in the meantime. They do occasionally replace components with equivalent Rust components from Servo, and that's so far been a great success, but it's still scary as all heck to take a software that millions of people depend on in their daily life and wholesale replacing the CSS engine, URL parser or media decoder in it.
Besides that, it's not like Chrome/Opera, IE/Edge or Safari are bastions of security. Users can't go anywhere that's decisively safer.
Plenty of warts in C++, but at the end of the day, along with C, it is the systems language that powers 99% of the world's computing infrastructure at any level that's not a CRUD app or a throwaway script.
Firefox OS all over again.
No. It's why I've lost interest in VR/AR software development for now; the real work to be done is in hardware. Even the latest HMDs are too low resolution and bulky for any kind of productivity gain. AR will be amazing once the tech is there, but the current design paradigm of "smartphone-strapped-to-face" just isn't going to cut it.
We're seeing a lot of industrial use of AR in manufacturing and industrial settings, though consumer use is still less common. If you have't come across it, this HBR article is one of the best public materials: https://hbr.org/2017/11/a-managers-guide-to-augmented-realit...
VR is certainly still emerging as a platform and trying to get out of its hardcore gaming and training/corporate silos. We really believe from early user data and some more advanced markets (e.g., China) that standalone VR headsets and great cross-platform social experiences will help it reach more users.
All that said: I don't personally know what the "instagram of VR" will be, though I certainly wish I did :-)
The only way we can really call today the "early stages" of either of these technologies is by imagining a future where they dominate. But that's the very thing we should be questioning. I note that pretty much anything 3D has a long history of being heralded as the "early stages" of a revolution that never came.
The most obvious example here is 3D video. We're at the tail end of a boom in 3D movies, a technology that now only gets applied to certain high-priced, effects-heavy blockbusters, and could well vanish. There was an even shorter 3D TV boom. [3] And of course there was the 50s boom in anaglyph (aka colored glasses) 3D. [4]
But previous to that there was the ViewMaster, which was imagined to have all sorts of potential. In WW II, the US military bought 100,000 viewers because they thought the magic of 3D would be better for training soldiers. [5] And this history goes back to at least the Brewster Stereoscope, which sold 250,000 units in the 1850s, and was also expected to be revolutionary. [6]
Given the many waves of hype in this era, I think we should be careful of thinking that we are on the path to some sort of destiny. Smart people have been fooled before. It's perfectly possible that this will be just another hype cycle whose main result is a bunch of dusty old hardware in the junk shops of 2050.
[1] http://www.augmented-reality-games.com/history.php
[2] https://killscreen.com/articles/failure-launch/
[3] https://www.cnet.com/news/shambling-corpse-of-3d-tv-finally-...
[4] https://en.wikipedia.org/wiki/3D_film#The_%22golden_era%22_(...
[5] http://legendsrevealed.com/entertainment/2015/08/24/did-the-...
[6] https://en.wikipedia.org/wiki/Stereoscope#Brewster_stereosco...
Eyes are definitely wide open!
The world is an extremely different place. This is, in my opinion, still very early days - by which I mean that we are still in an exploratory phase.
We've never been closer to having the technology necessary for it to take off - resolution was always one huge blocker, portability, weight, 'culture' in terms of computing being something everyone has access to, GPU power, battery, etc.
As far as taking off, we might be close, and we might be far. It's tautological to argue that VR is coming soon because we're close to the level of technology that means VR is coming soon.
And it may never take off. Maybe we'll get to a particular level of technology where everybody is wowed by the tech but in day-to-day use nobody actually cares. That has happened over and over again with 3D technologies. We are definitely in an exploratory phase, but a great number of explorations don't end up going anywhere interesting.
I'd suggest domination is not a prerequisite for success. Let's imagine a scenario where only 5% of the world population uses VR or AR on a regular basis. That's still a market of millions of people, but it's not at the level of world domination. Not every new technology has to replace the one before, it's far more common for old and new technologies to co-exist.
The ViewMaster is a fine example there. It's a neat novelty technology! I loved mine when I was a kid. People still love them today. Is it successful? Sure. Do millions of people own them? Definitely. But in retrospect the DoD purchase of 100k viewers doesn't seem like the early stages of anything.
Or think of satellite phones. The Iridium program was massively hyped, and it was a technical success. But commercially it never took off. Satellite phones are still available today, and there are circa a million subscribers. Was the 1990s hype the "early stages"? Again, in retrospect I'd say no.
So sure, domination isn't necessary for success. But I think future domination is necessary to retroactively justify the hype and investment of our current era.
As for investment, you have a point. Perhaps the level of investment will prove to be a mistake, but at this point in time we don't really know how big of a demand there will be for VR and AR. I'd struggle at this stage to predict how it will end up, but what I can say is that I can see evidence supporting the prediction that they'll be mainstream, as well as evidence supporting the prediction that they'll be niche. We probably won't know with any certainty until after the second or third generation of current devices is on the market, as that's the point where they're likely to take off commercially if they're going to. Speculation before that point can be fun, but I don't think it'll change the outcome.
Especially so given that hyping something is "to promote or publicize (a product or idea) intensively, often exaggerating its importance or benefits". It's not a neutral act. And in the commercial context, the purpose of hype is generally, in one way or another, to put money in the pockets of the person doing the hyping.
Depends who's doing the hyping. If it's a potential consumer, then hype is derived from excitement, and the same rules that apply to the justification of excitement would apply to the justification for hype. On the other hand, if it's a company hyping their products, then they may not be led by their feelings, but I don't see the harm in it either. What do you lose out on if a company is hyping their products?
What's more interesting to me is that you're implying that this hype (a.k.a. advertising) is having a negative impact on you, even if you don't intend to buy the product yourself. What's the worst case scenario here? That some investors make a bad investment?
If you would like to make it my job to educate you, my casual consulting rate is $250/hour, 10 hour minimum, paid in advance to MBO Partners in Virginia. Until then, adieu.
Forgive me if I pass on being enlightened on something as trivial as hype. I hope you teach something more substantial to those clients paying $250 an hour.
The #1 problem with this vision is the resolution of the HTC Vive and Oculus Rift. I haven't tried the HTC Vive Pro yet, but I'd say it's got to be about twice the resolution of the HTC Vive before I'll be able to actually use BigScreen the way it's meant to be used.
Once that happens, all bets are off.
1: http://research.nvidia.com/publication/near-eye-light-field-...
I know some people claim pornography has been a driver of technology, but necrophilia? really?
Surely the more normal version is the one GP was talking about.
So whether there are or aren't, there's clearly potential. It doesn't seem right to dismiss it just because it hasn't taken off yet - it's extremely early days.
The model is built in Revit and using this method allows for easy fly/walkthrough and spotting of some issues that might not be as visible in the 3D viewer mode on your screen.
And well, it is coming sooner or later. If as a browser, you aren't ready when they actually do, you might see an entire industry standardizing away from you.
https://www.npr.org/sections/alltechconsidered/2017/03/18/51...
As the tech becomes more commoditized it becomes viable for other businesses that don't spend as much on training.
Joining up with the mixed reality team is more about how the Mozilla staff working on Servo will be focusing their time and some of the desire we have for things we want to do for mixed reality that are a little invasive and early-stage of the standards process to be experimenting with inside of existing production browser engines.
With regards to a full browser, there's still a pretty big gap before Servo could stand on its own in a browser (https://github.com/servo/servo/wiki/Remaining-work ). Filling that list out is not a higher priority for the Mozilla staff over getting the stuff done we need to experiment with delivering the web on VR & AR devices.
So building a fully functional browser is not higher priority than getting the half-baked browser working in a niche environment? I'm half joking here, and half serious. How is a browser supposed to look differently in VR? And why does it need to be aware of the fact that it's in VR? People are still developing interaction and navigation methods in VR.
Great question! Some of the best early work on this was done by ex-Mozilla, current-Google employee Josh Carpenter - check out:
http://www.joshcarpenter.ca/declarative-3d/ http://www.joshcarpenter.ca/vr-browsing-explorations/
There's a lot we can do that is even beyond these early explorations to deliver new features to developers to experiment with and users to try soon via Servo, alongside GeckoView in new VR/AR-focused browser products.
If you take a look at the link I provided above on remaining work and rough estimates to get things just working in Servo (and not FULLY web compat), you're looking at an effort of several years for the entire team, and that's assuming the web platform both stayed still and went ahead and finished writing all the tests and specs for everything on the web that is today neither tested nor fully spec'd. I don't see that in the cards for 2018.
Not passing judgement on anything with the above statement. Just trying to make sense of it all.
But I don't think we've ever had a concrete goal to have a standalone Servo browser. Parts of servo in firefox? Sure. Trying out new ideas? Sure. Embedding servo a la electron or WebkitView? Sure. But Servo as a standalone browser has always been a "maybe someday" thing -- we've never worked against it, but I don't recall us ever explicitly targeting it, though I think most of us have always had some hope that we'll reach that point eventually.
That is, continuing the Quantum work until there is either no C++ code left, or the C++ code left can turned off via config option ("prefer perfect security over perfect compatibility") while still having a browser that works on the vast majority of the web.
That said, this news do not sound good to me. Again, this is from a great distance, but from here, taking a future-focused team with great respect from the community, a team with an important and invaluable mission and that has successfully delivered over years, and shifting it (or forking it) towards complete uncertainty feels like a dangerous move that reminds me of the phone OS efforts.
Good riddance and may your vision be right. We need someone with Mozilla's goals looking deeper into VR so that we don't end up in the hands of the Big Corps in the near future, that is for sure, no matter if VR ends up being niche. How important that technology will be for the masses is yet to be proven and that's why I fear for the web, the battle proven real thing that faces the greatest threats from closed silos today. I don't feel it should be taken on equal terms with what is still fictional speculation. It needs full energy to make it to the future in good health. We already failed to deliver a beautiful, open and user centered internet to the future, we can't loose the open web.
I still have a hope that Servo will be used in a next-generation Electron competitor, as that's what I thought would be the best use of Servo (in the short term). Hopefully the developers can keep this use case open by maintaining the code that allows Servo to be embedded as a component in other projects.
That's what's already happening with Project Quantum, and that project is still ongoing (as far as I'm aware).
https://wiki.mozilla.org/Quantum
https://hacks.mozilla.org/2018/01/firefox-58-the-quantum-era...
This is extremely disappointing.
I think the amount of users you can win back from Chrome with a Servo browser is much larger than any VR audience you can hope to gain in the near future. I simply don't understand how these things cannot be done in parallel. Mozilla has hundreds of millions in cashflow, right?
Looks like Servo will be another exciting Mozilla project that will be prematurely dropped after a ton of work and hype-up. RIP Firefox OS, Persona, Thunderbird, etc... :(
A Servo browser product is not possible in the near future. That is very much a far future thing. Web compatibility is hard.
Servo has a lot more potential in platforms where you don't need to support the _entire_ set of browser features -- like as an electron replacement, or a platform for something like webVR, or Android embedding (which will probably come out of the VR work as well).
and an impossibly far future if the Servo team is now refocused on VR, no? how likely is it that these hard problems will be solved by fewer engineer-hours?
perhaps there's some external backer willing to bankroll VR and Servo is just an easy expense to curtail.
Focusing on VR means that we may not be working on some kinds of webcompat things (the standard example is "IE6 table layout quirks") but there's plenty of webcompat work that needs to be done otherwise.
Yes, not everyone will be working on webcompat, but this has never been the case anyway.
if a new browser in 2019 could not properly render an archived 1996 Geocities webpage at time of launch, i don't think many users would care or even notice. there are almost certainly legacy bugs in Firefox that have been open for over a decade with little activity and yet it's in production.
Seriously, web compat is very tricky.
We're still working on standards compliance, but we may focus more on the bits that are actually useful within VR.
Having folks spending months implementing the corner cases of standard might not researching and finding creative ways to use Rust to implement large systems.
Standards compliance probably follows some exponential curve wrt energy and results.
It's too clumsy. If you aren't gonna walk with it then it's 360 display that makes you vomit, and if you are gonna walk with it not every consumer has huge empty room where that's feasible. Most of us have couch or desk at home where we consume media, not empty warehouse. I predict whatever Mozilla thinks they are doing it will fail for the same reasons Google Glass failed spectacularly or why after years already spent on VR I don't see anybody outside koolaid-drinking techies owning one set.
Mozilla has one good consumer project that can realistically be completed (browser in type-safe language) and getting the team behind it bogged down with tech that's immature and doomed to failure is unfortunate.
Distraction does seem like a major risk but I like the way they’ve been pulling pieces over to Firefox. It’d be really nice if that model worked so they could avoid getting as bogged down in web compatibility.
So, for Servo to catch up to the current state of Firefox and to then also keep pace with new web standards, it would have to be developed much quicker than Firefox is being developed. So, that would require more than double the development capacity that Mozilla currently has, as they can't drop Firefox development either.
There is some points that could alleviate this: <ul> <li>A smaller project grows quicker, <li>it being written in Rust might speed up development in the long run and <li>several components are now shared between Firefox and Servo, meaning that the development and maintenance work is shared as well, <ul> but it still is far away from realistically completable.
Which is also not what the project is meant to be. It's a research project. Trying out new things, creating components that can be used in Firefox, and doing experiments with VR or similar are exactly what it can be used for.
And you don't have to repeat 20 years of Firefox to complete Firefox, a lot of it were dead-ends like XUL (themed GTK would do), a lot of it were non-essential features like Pocket and a lot of it were rewrites and mistakes that they won't have to repeat and like you said being written in Rust should speed things along. Lastly, I don't care if Servo is released standalone or if Firefox gets replaced by Servo like the ship of Theseus, all I care about is browser that is less explodable, otherwise I'll continue on Chrome.
Pocket required essentially no work. They bought an existing service and stuck a fancy bookmark for it into Firefox.
And well, I'm mainly just repeating what Mozilla devs have said. They are deeper into the matter than both us, they can better judge just how much work it is. And it's easy to forget just how complex web browsers are. None of the major browsers use a browser engine which's development started in this millenium for exactly this reason.
I'm also not aware of Chrome being less explodable. I'd say, it's more explodable with its malware-filled extension store, default unencrypted sync service and annecdotally a vulnerability like this [1] being left unfixed for years and careless behaviour like this [2].
[1]: https://github.com/anttiviljami/browser-autofill-phishing
[2]: https://www.wired.com/story/chrome-yubikey-phishing-webusb/
Firefox used to have a worse security architecture, but it's essentially equivalent now. The only real difference that I'm aware of, is that Firefox by default groups processes for tabs, meaning if a webpage manages to exploit a major vulnerability in Firefox to gain control of the process it's being executed in, then it has access to 1/4 of your tabs and therefore might potentially be able to steal sensitive data, whereas in Chrome it would then also have to exploit a vulnerability in the OS to do that.
Or execs are driving a bus into the wall. It wouldn't be the first time that happened for sure.
> I'm also not aware of Chrome being less explodable.
It used to be. It only took them 10 years to enable sandboxing.
> with its malware-filled extension store
And no rational person installs extensions from it for that reason without first checking who stands behind the extension.
> then it has access to 1/4 of your tabs and therefore might potentially be able to steal sensitive data
So essentially not equivalent.
This is the most important missing piece, are their bugs for this?
Until we have html/dom textures in vr I just don't see webvr being very successful.
Just like the headset isolates you from the room, the lack of normal web content in vr/mr is very isolating.
Anyway, I was just wondering if the Servo team had seen the work Elevr had done on basic interaction design? They lost funding last year but I thought they had a ton of work done.
That's being a bit liberal. Sure, [AV]R is going to be important in the next few years, but it certainly won't be mandatory. People will still be using flat monoscopic (not stereoscopic) displays.
The work Mozilla has put into Servo is clearly very useful right now. There is still a lot of work to be done on Servo, and on Rust itself. Are you really that surprised that some of us are wary of splitting that team's priorities to work on something like [AV]R?
Or not. The amount of hype and investment isn't a reliable measure of future success. We've been here before. The moment when mass adoption is just around the corner.
Some things are conceptually exciting, but disappointing in practice.
You’re still not as fast or battery-efficient as Safari on macOS, so for me, Mozilla has more work to do on its core products before jumping onto a new tech project (like Firefox OS) that doesn’t even have a clear use in the market.
You’re not Sony. You’re not Facebook. Focus on what you’re good at now, let those guys find the market and the killer apps, THEN jump into AR.
You still have more work to do where you are. Tell the product people to shut up. FOCUS. Fewer features, better quality.
Microsoft's Chromeffects did this in 1998. It was pretty amazing, but alas, got canned less than a year later.
I fear that ship has long sailed.
For those of you simply dismissing whatever's happening because VR has been hyped before and it didn't pan out this time, I'd highly recommend this post: https://signalvnoise.com/posts/3124-give-it-five-minutes