Inside Magic Leap
forbes.com
forbes.com
>When it arrives–best guess is within the next 18 months–it could usher in a new era of computing, a next-generation interface we’ll use for decades to come.
...which may be a pretty valid statement or sentiment overall, but later in the article, we get to the crux of the "potential" future of the device / platform:
>Eventually Magic Leap sees its greatest impact in business applications, especially medical imaging and retail (imagine “trying on” garments at home, seamlessly). But as with most technologies, entertainment offerings will lead the way.
I might be picking on a low hanging fruit here, but seriously, one of the compelling reasons to look forward to Magic Leap is so it can have a tie in with the Home Shopping Network or QVC or Nieman Marcus?
The more and more I see the little teaser images (and the occasional demo reel like the office game) I'm cautiously suspicious the technology is perfectly reasonable but figuring out great content for the platform will be the difference between whether it will align with the hype and promotion or be closer to the storyline of the Nintendo Virtual Boy.
I'm just frumpy after years of seeing the difference between renders for PR purposes and actual in-game / in-environment footage as a player / user, which, I do think as tech has improved, isn't happening as much anymore in commercials and whatnot, which I like. This particular situation and eventual roll out will prove a lot I'm sure, one way or another..."soon-ish" as he says...
I personally think this would be really cool, and would facilitate entirely new modes of interaction with people in this field e.g. designers and sales workers. Imagine being able to connect with a clothing designer remotely, and have them model you in a virtual outfit before a mirror, with an practically infinite vocabulary of color, texture, and pattern at their disposal. A 2-way conversation with a design professional with a vastly accelerated prototype-feedback loop.
No, I think this would be totally awesome.
Also people are very "forgiving" if things seem to wiggle a bit in the world but looking at the Google Tango device this weekend I got to see some of the challenges of keeping registration of the pixels high. Its frustrating if your ruler end point is "jiggling" a bit close to the spot where you want to start a measurement.
So without good registration you get eye fatigue like you used to get with pilots flying long range on bumpy flights constantly looking at instruments. Only in the case of pilots they can look outside to the clouds (which don't "bump" in space).
+ To have a cat even if you can't feed it or clean up (elderly).
+ To find out how it is to have a baby, how to feed it, how to care for it.
+ Dating first with an avatar before going for the real deal.
+ Searching for things without having to open cabinets or boxes.
+ Colored lights etc will be outdated. Colors, paintings, candles, can all be virtual.
+ Augmentation including audio can give a voice to your pets, to inanimate objects, even the food on your plate. Your plant will tell you that it's thirsty.
Also I think screens are definitely not outdated. I look forward to have a screen the size of a wall and "wormhole" functionality so I can virtually couple it to a similar wall somewhere else. Then I will feel much closer to my grandma for example. Or my wife to her loved ones at the other side of the ocean.
Maybe I'm just weird or old-fashioned.
I know my grandma didn't want pets because she is too old to bend over.
I know people who carry a sack of flower rather than a baby.
What I think however is that we have to consider what it means to interact with the "real world".
If we filter out advertisements through augmented reality, will it get us closer to the real world?
If we can remove visual clutter, noise pollution, will the people without glasses be bombarded with spam they can't filter out?
Moreover, will the world become more real by knowing things about the meat on your plate that you currently can ignore? Will we be able to effort not knowing things?
That Terminator tech - seamless glasses that let you assign and view little annotations on things.
# Conversation with John Smith
## Family
* How is his daughter doing in ballet? (guaranteed, last discussed 1m3d ago)
* Was his wife's birthday last weekend? (70% certainty, Facebook feed. [Review post])
## Video games
* What is his current rank in League of Legends? (guaranteed, from your LoL friends list)
* Has he tried any new champions in League of Legends? (the answer was "yes" 1m3d ago)
## Shared occurrences
* Relate your experience with your participation in a hackathon last weekend.
o Has he been to a hackathon? (shared interest in software development)
o Does he have any techniques for making travel bearable? Has he ever been to upstate New York? (he travels often)
* Talk about your upcoming shared trip to Canada.
o Has he previously visited Canada? (unknown)
o Does he have his ski equipment ready? (the answer was "no" 1m3d ago)
o Is he ready to have a good time? (the answer was "yes" 1m3d ago)
o Relate an anecdote about the last time you went. (you have not already done this)But in reality what it really takes to disrupt an industry is both a disruptive product and a really compelling mass-market price point. One of the reasons the Segway PT failed is it was $6k to buy. That severely limits the addressable market to only the rich and those that really want a new form of transport.
This is the same problem Magic Leap and Microsoft's Hololens have. If the product is more than say around $700, it's not going to be a mass consumer product, and it's not going to disrupt entire industries. The iPhone is a good example of a product that is sold for about the maximum price point a user can tolerate in most countries, and even then a lot of iPhones are sold in installments to bring the upfront cost down.
I find it hard to imagine that Magic Leap can hit that entry level price point in a Version 1 product, and if they can't then there will be quite a few big tech companies gunning to catch up with their technology by the time they hit a V2 to erode there exclusivity. Just look at how Google turned Android around after the iPhone launched to go with full screen touch just 6 months after the iPhone.
In any way, I can't see this not being a technological advance once we'll have played with it as makers for a decent amount of years.
Ask Pixar how much time goes into draping a single VR garment on a single body. It's hard work, regardless of how good your imaging tech is. The ongoing hurdles are about modeling, based on cloth properties and personal anatomy. The margins on "try on at home" absolutely don't justify modeling every singly piece of clothing into a technically-accurate VR solution, and if you do anything less people will return your produce when it looks nothing like the VR visualization.
Asserting that a good VR technique will justify real-world modeling is downright stilly, and it leads me to ignore non-tech assessments of whether that project is worthwhile.
And yet... Check out the bodies displayed at 2:08. A good range, but nowhere near the range in effect for real people - and at least half of them look noticeably wrong to me.
The cloth displays aggressive horizontal wrinkling, in a way that makes no sense. Check out the center-most figure, whose shirt curves in below the stomach with extensive wrinkles that gravity should remove. All of the figures display confusingly broken folding on the horizontal plane - the leftmost model stretches the cloth across the chest, then displays bizarre smooth-to-wavy patterns at stomach height as you move from front to side. And as far as I can see, all of this is weak to multi-layered, stiff, or tailored cloth.
So yeah, I appreciate this, but I think it's woefully inadequate to modeling even cheap, popular clothing. That's going to take a few more breakthrough as large as this one at least, so Magic Leap's VR abilities aren't nearly enough to justify this business model.
Tends not to be a linear progression at all; if the attention of more than a couple researchers is applied to any of these things then they grow very quickly - couple this with improvements in parallel processing power and it becomes quite viable. (see: physically based lighting, rigid body -> full crumple simulation, fluid dynamics, Photoshop's "smart fill", or really any game tech)
Apropos of nothing: Same applies in terms of hardware - what we can do with light field cameras now was science fiction just over a decade ago, but within a couple years of crossing a research tipping point we went from 10x10' camera arrays to <$100 hand-held commercial products.
source: graphics tech and SIGGRAPH nerd for the last 23 years
actually there was an extremely slow build up of this tech, with a sudden and rapid escalation - but between 1908 and 2010 was a very shallow incline :)
People so often underestimate the difficulty of productizing significant UI changes, not realizing that getting content to migrate to the new shiny will be at least as hard as getting rid of Flash on the web.
Bullshit. You're going to model and diagram every part and repair guide from every production model vehicle ever?
Magic Leap would need a software development industry unto itself to build the solutions being batted lightly about in the OP like pingpong balls
Now, whether or not they are actually going to be the biggest thing in graphical interfaces since the flat screen is another question...
They aren't going to beat Oculus/FB, or Sony, or HTC.
They don't have a product; they clearly must have some incredible tech demos, and a wizard working VC
I remain highly skeptical.
In my experience, the problem is more in how the model is organized; having ever screw or washer as a separate piece of geometry causes problems (3d tools have trouble with very large geometry counts even if the poly count isn't high). CAD data also doesn't work very well with non-hard materials like upholstery; stitching is likely missing, as well as things that make cloth/leather look normal like pinching or gravity pushing on things.
Maybe Magic Leap has the same types of engineers [does Graeme have the same reputation as hyper productive dev?], but there are reasons Carmack has this reputation [1].
And I've known and worked with people like this. People who are utterly relentless. Insanely productive. And honestly? I would always bet on them. Every single time. It's just a major key that I don't know anything about on the Magic Leap team.
[1]: http://venturebeat.com/2015/01/27/how-john-carmack-pestered-...
I think there's some practical considerations here with the "magic engineer" who can do anything quickly and, of course, the laws of diminishing or even negative returns outlined in books like the Mythical Man-Month. Even the impressive Santa Cruz is years out from being sold. Graphically it cannot compete with the PC-based systems it would ultimately compete with.
https://en.wikipedia.org/wiki/Oculus_Rift#Initial_prototypes
Say what you will about Microsoft, at least they already have a platform on which this "new era of computing" can be based: Windows. That is to say, it makes tremendous more sense to evolve from what we have now into the "new era of computing", than to write an entire platform from scratch, which is what I assume Magic Leap is trying to do.
Magicleap = supposedly infinite (or they don't exist at all?)
Hololens says they have 2.3 million “light points”. I don't know much about light points other than a few paragraphs, but I wonder how Magicleap actually measures light density and how/if it compares or correlates.
Regardless, GPUs will likely limit the render resolution for the first one or two products. Upscaling might be possible, though.
I presume a "light point" is some sort of shaped structure on their glass which allows the scanned light to escape to the eye when requested. And in theory, might be a place where they can do interference cancellation of incoming light to darken points. All their hype probably points to metamaterials-style shaping of light.
Remember that pixels aren't rectangles (hence why scaling images larger always looks more blocky or blurry than native res), they're point samples. So a "light point" should be equivalent to a pixel in the simple case. Whether or not they can bend light between points to make it "infinite", who knows. Any of that is always going to be limited by the resolution that the GPU can crunch in time.
Just how impressed are they?
Let's see.
Magic Leap = 4.5bn valuation
Snapchat = 20bn valuation
So if, as the article states, ML could disrupt the $160bn LED screen market (among other markets), the valuation would reflect that.
Presently, it does not.
Either collaborate and share or be secret and reveal, but this is the worst combination. If there was a way to short this company I would do it.
"Throw out your PC, your laptop and your mobile phone, because the computing power you need will be in your glasses, and they can make a display appear anywhere, at any size you like."
Yeah sure, they not only made a new display tech but integrated the computing power of desktops or laptops into glasses. I wonder how they cool the GTX 1080 cards on your head...
But might was well stay hopeful for as long as possible.
Problem is that would require at least two huge leaps in technology one in display technology and one in compact computing power, probably more.
Cloud gaming has not done all that well; it's right at the edge of feasibility due to latency. VR moves it back off the edge to "unquestionably infeasible". If VR gaming were to take off and become a dominant paradigm it would make cloud gaming even harder to justify.
Pseudo-real-time gaming is possible, but only for regional servers. Live global gaming is off the table permanently unless known physics changes; speed of light delays are larger than human reaction times. Right now, the best we can do is predictive techniques and continental servers, which have barely enabled MMO shooters.
VR gaming worsens the problem - 80ms delays in VR are nauseating and game-ruining. Global latency isn't a solvable problem via money or devotion.
There is a limited useful spectrum that's shared between everyone in the area. You can't just throw more bandwidth at the problem.
I was completely skeptical at their claims with respect to latency. I never believed they could get the service to work, and for enough people to make it profitable. It appears quite a few people were able to enjoy the service at decent enough framerates. Which is surprising.
A few weeks ago my cousin ran games at 1080p on an EC2 GPU instance and used Steam's In-Home Streaming to play remotely and it worked perfectly fine.
I think the latency is now there, if you have a fast connection and live relatively close to the datacenter.
It would certainly be a much more complex solution that would still have inferior results compared to computing everything locally (just like cloud streaming now). Just that it's not "change the laws of physics" impossible to get to a workable implementation.
See also John Carmack's comments from 2013 on latency mitigation: https://web.archive.org/web/20140719053303/http://www.altdev...
(It's not actually required, but the efficiency loss between H.264 and HEVC for 4K is pretty massive.)
Also considering how litigious/secretive things are — it must suck to work on something for so long and be contractually obligated to avoid talking about it in public.
http://uploadvr.com/magic-leap-how-it-works/
Magic leap is a pair of glasses you wear on your face that displays AR imagery through a clear glass lens. There's a cable leading from these glasses to your pocket where you have a tablet-size device doing all the heavy lifting, most likely using mobile SoC's. Pretty sure you can't run a desktop CPU and GPU on a pocketable device.
You don't get 1080gtx performance or anything remotely close to that with a setup like this. Its mobile-level graphics. Oculus's recent showing of its mobile SoC-based Santa Cruz prototype shows fairly poor graphics compared to PC-based VR like the Rift or Vive. I imagine the Magic Leap has less pixels to push considering it isn't doing any backgrounds, but still, those very impressive marketing shots of high resolution dragons flying around and interacting with you might not really be possible without a lot of compromises on graphic quality, fov, framerate, etc. Worse, no one wants to talk about transparency, especially in well-lit rooms. The few hands-on Hololens reports we've gotten make this out to be a big issue, as well as fov, and fundamentally the Magic Leap and what hololens is using could be the same or very similar technologies.
My gut feeling is that no one has yet to handle the transparency issue (and this is why MS can show off the hololens with confidence) and because of that it makes sense for Magic Leap to keep its product secretive. The tech press was not too kind to Hololens prototypes due to transparecy issues and Magic Leap doesn't need that kind of negative press as it continues to fund-raise and develop its product.
If we want current gen graphics we're more than likely looking at a PC backpack solution. More than likely we'll probably be looking at some low watt solution, like the 35W AMD chipset in the Solus Q and accept a lower level of graphic quality, but still well above mobile standards. This may involve wearing a battery belt or battery hip-pouch for extended use. For lower run times the battery can be part of the headstrap with no need for anything to be pocketable.
The Oculus et al need to render a rectangular matrix of evenly spaced pixels. If you want a larger field of view or higher resolution this comes with an exponential increase in number of pixels. With a high frequency scanning display and eye tracking you are not locked to a static spatial resolution (see https://en.m.wikipedia.org/wiki/Foveated_imaging).
There's a whole bunch of other problems when it comes to a render pipeline for something links this, but at least on the physical front it enables super high perceived resolution with less overall points of light to render.
It was interesting to hear Carmack last year talk about re-introducing interlacing as a way to reduce latency.
Although, I'm just reinforcing what you're saying =)
For VR, the consensus I've seen seem to push for multi-sample AA > full-screen AA > temporal AA (I feel like I've even seen no AA is better than temporal AA). I'm not quite sure if it's performance, architecture (they really prefer forward renderers instead of deferred renderers for low latency), or aesthetics. In the little bit I've done, when just playing with knobs temporal AA looks better to me. Without AA specular highlights are way too distracting and pop too much.
This reference[2] talks about the extra velocity buffer needed for temporal AA and the fact it tends to over-blur which can fuzz out fine details (low resolution is a touchy subject in VR).
There's a lot of moving pieces when making decisions. In talking about interlacing, he was ideally talking about not having to create a whole image buffer before sending data to the hardware--just rendering parts (well, scanline) of the image that moved.
[1] https://youtu.be/gn8m5d74fk8?t=10m54s [2] https://www.youtube.com/watch?v=Xk3WUk5T2TQ&feature=youtu.be...
How does that work? Shouldn't it be just linear in FOV? Or quadratic if you are talking about solid angle? This doesn't seem like one of those things that grows exponentially.
'Enough' is retina resolution, but only on the fovea, and way less the further away you go from the fovea. That can significantly decrease the amount of work needed to render a scene.
Moreover, color-wise, you don't need color except for the center of your field of view (you need cones to see color, and cones can only be found in the fovea).
Building such a screen probably isn't that hard; you would need to build new production lines because everything out there is built for rectangular grids of equal-sized pixels, but that probably is 'just' a matter of investing money and time.
Writing the software to efficiently draw scenes on such a screen probably is a bit harder; there will be lots of optimizations that need to be discovered (what's the equivalent of Bresenham line drawing, for example?)
The biggest challenge will be to keep the high-resolution part of your screen centered on where the retina looks. That requires eye tracking and some way to move your screen. Saccades move at hundreds of degrees per second; you would have to match that.
If that's what they are building, and they succeed, I think it will take over the world, even if they initially sell it at $5000 an eye.
- FOV is apparently their big advantage, along with focus zones
- They could do hard occlusion with an LCD mask. Or they could punt til v2, and soft-occlude with blur using their stacked diffraction plates.
Unfortunately, you cannot easily achieve "hard occlusion" with an LCD mask. Pinlight display is a project that did explore that idea: http://pinlights.info
LCDs can attenuate individual rays of light. However, the images we see of real objects in our environment are generated from light in the form of planar wave fronts. So, if you have a near eye LCD micro display and you turn on a single LCD pixel in an attempt to block light emanating from a certain point on a real object, most of the light emanating from that point will simply go around the LCD pixel, enter the pupil and for a point on you retina. The total brightness of the image will dim a little dependant on the size of the LCD pixel, but not by much for a single microdisplay pixel (i.e. a few micron pitch).
The Pinlight papers do talk about ways around that issue, using LCD diffraction pattern masks, but I still think the residual diffraction and reduced overall lens transparency associated with the LCD micro display makes the approach an even bigger challenge.
Could you emulate that with a transparent LCD display? Yes, but that display would need to be large and about 1.5ft from your eye. Not really practical for a near eye (head mounted display).
If you try to accomplish this occlusion directly with an LCD microdiplay (such as those found in many projectors) in a near eye application, at best you end up with a blackish smudge with very fuzzy edges. To see this in action, look at Figure 11.D in the following paper:
http://www.cs.unc.edu/~maimone/media/pinlights_siggraph_2014...
>[0216] Such may be used to cancel light from the planar waveguides with respect to light from the background or real world, in some respects similar to noise canceling headphones.
http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=H...
> http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=H....
I believe this particular excerpt is referring to the use of the layered and controllable DOEs to curve light projected from their scanning fibers that ultimately forms the virtual object image on the retina, rather than using the DOEs to attenuate ambient light (see my reply to Kelsolaar above).
However, if I'm interpreting their literature correctly, you are correct in that they are able to use (at least a portion of) these DOE layers to generate the moire patterns that will ultimately be capable of form black/clear image masks.
On the other hand, I believe ML is proposing to use a gratings to display black/clear images. In this setup, the incident ambient light wavefronts are selectively redirected using bragg gratings (why they keep saying Braff gratings" in their literature is beyond me). This technique has been around for many years and is still captivating to watch even in this day and age - see:
https://www.google.com/url?sa=t&source=web&rct=j&url=https:/...
The realization that this affect retains its "sharpness" for near eye applications in concert with (I believe) the realization that you can utilize the electonically-controlled diffraction grating layers you already have in your device (for the purpose of controlling wavefront curvature to support eye depth accommodation) depth is really quite clever. It's truly an exciting prospectct!
> You don't get 1080gtx performance or anything remotely close to that with a setup like this. Its mobile-level graphics.
A.) the pocket device is the light source array that feed the scanning fibers. A processor drives each light source linearly. Neither the light source nor the driving processor are likely to be head mounted for quite some time, but I don't see that being an issue given the dramatic improvement over existing interfaces (i.e. I'd rather have a pair of glasses with a tether than a helmet any day of the week);
B.) In light field VRDs, resolution is not limited by active matrix array addressing as it is to a degree in LCD displays of mobile devices. Instead, resolution primarily depends on how quickly you can resonate the tip of a scanning fiber and concurrently how quickly you can pulse that fiber's light source in a synchronized manner (each pulse matching the bit color/intensity of the intended pixel);
C.) Multiple fibers form a composite image;
D.) Maximum resolution is only needed in the foveal region (i.e. a few deg about the center of user's eye gaze at any given time), so eye tracking can be utilized to minimize processing. The remainder of displayed content is low resolution, albeit still critical, peripheral imagery.
My team & I have been batting around with a slightly different approach to the light field VRD, which we're thinking about simply releasing as a FOSH project.
More details would be fantastic
The resultant display has an inherently smaller exit pupil, but we'll layout a method to actively expand the exit pupil, enough to support typical maximum eye rotation range and minimum eye pupil dilation (~2mm) without vignetting the virtual image with natural eye movements. The benefit of this method is virtually unlimitted FOV.
Certainly still in development, but we think releasing what we've come up with thus far into the wild as FOSH and seeing what the crowd can bring to the design is a fun idea.
I'm currently working on a simple "Hello World" write up that will explain the device components, some basic plans for prototyping done to date and the development hurdles that still need to be overcome. I'm planning to use this to introduce the project. It's definitely not an easy project, but I don't see any component that can't be fabricated by a crafty homebrew engineer.
One of the researchers posted a 30ish min video on the hardware inside the Hololens on YouTube. It's this HPU / sensors that Microsoft is licensing to the hardware manufacturers for the upcoming VR and AR headsets. We'll know more December 8th, and after their hardware expo later this year.
But their later patents showed whole arrays of these fiber projectors. They are probably limited by the optics in-between the fiber and the eye and by the display controller bandwidth more than anything.
Ugh. I just imagined a room full of people sitting around a conference table with VR headsets on. Not a pretty thought indeed.
BUT
I'm still pretty weary of billions getting pumped into a business that remains unproven.
+ Maybe development is very expensive + Maybe the goggles cost to much + Maybe they just don't have quite enough content to justify the high costs + Maybe the experience 'isn't quite there'
So many times these 'wow' toys have proven to be near-duds when they actually hit market reality.
I can't wait to try it, but I'm also looking forward to seeing the reality of it.
You can see this as and extension of the skateboards vs scooters vertical stick argument.
PS: For comparson 17.5 million cars where sold in 2015 and 12.5 million bikes with 20+" wheels.
I don't know if this is really true. I saw a lot of hoverboards on my university campus last year. I don't see nearly as many this year. I'd like to see sales data for at least another year before declaring that anything is becoming increasingly widespread.
http://mashable.com/2016/03/16/segway-hoverboard-patent/#ZFs...
Anyway, I think we are just seeing the hype / adoption cycle: https://en.wikipedia.org/wiki/Hype_cycle Basically, when nobody has X a lot of people get it all at once, but soon enough people have it and so they wait till the old one breaks before getting a replacement. Still if long terms sales stabilize at say 1 million units per year that's still significant.
>The centerpiece of Magic Leap’s technology is a head-mounted display, but the final product should fit into a pair of spectacles. When you’re wearing the device, it doesn’t block your view of the world; the hardware projects an image directly onto your retina through an optics system built into a piece of semitransparent glass (the product won’t fry your eyeballs; it’s replicating the way we naturally observe the world instead of forcing you to stare at a screen).
This is purely my impression, but reading the line that says "the final product should..." leads me to believe they still have significant development hurdles that have been unconquered thus far, and being coy about the release window (the journalist speculated 18 months, the CEO just went with "soon-ish") encourages me to read between the lines. Others may interpret the line differently - that's fine - but I'm simply sharing my impression. Lots of things "should" work the way their visionary has in mind...
Isn't this exactly how every display works? Possibly sans glass even...
From the write up that was posted here weeks ago, it looks like the technology is really really interesting, but not for the reasons this author thinks. projection through optics isn't really anything new; projecting individual light points through a fiber cable with an oscillator at the end like one of those spinning LED clocks[1] but on your face is pretty cool though...
they work away from the public for product they don't know if the public want. it's a bet, i hope it will pay off.
Considering the hundreds of people they've hired and their partners (Disney, ILM, Weta) They are launching with applications already, a bit like a games console more than anything.
That doesn't mean that it is wrong, necessarily, but that it isn't always exactly applicable.
I mean, if you read the article they claim they're doing Agile-hardware instead
Anecdotally I've heard that lots of VC firms passed on all three during their very early days, but I'm not convinced that those are good examples of completely new markets.
We're already dealing with issues of sensory and information overload without having this layer of immersion.
[1] http://www.sun-sentinel.com/business/careers/fl-magic-leap-n...
At the time the Mayo case was decided, there was some uncertainty over whether it applied only to natural principles (laws of nature) or more generally to patent eligibility of all abstract ideas and general principles, including those involved in software patents. The Alice decision confirmed that the test was general.
https://www.linkedin.com/pulse/patent-eligibility-criteria-l...
[1]: http://gpuofthebrain.com/blog/2016/7/22/how-magic-leap-will-...
Don't get me wrong, this could be incredibly cool in a vacuum. But reality often has different needs.
I hope I'm wrong.
I'm not saying they're not making an awesome product. I'm saying it's possible not enough people will want to buy it in order to justify building a factory and campus.
It's not my money though and I've never seen the product. This is obviously a HUGE gamble that will either be a raging success or will become a joke about the follies of venture capital (i.e. WebVan 2.0).
The funds need to make big bets, this one looked good: best tech from UW prof, a founder who's sold a hardware co for >$1B before.
It depends very much on execution: polish, price, apps.
Magic Leap itself does seem to be doing amazing stuff, given how much money has been pumped into it. At least, I hope that's the case!
"I'm doing the thing that scares me the most every single day."
Like what? Browsing without an adblocker?
MLeap is the only one I know of which has raised the $ needed for consumer-level polish and price. Probably because of the mini fiber-scan display and diffraction zone plates for natural focus. The other major pieces (waveguide or holo lens, IR TOF camera, SLAM, eye tracking, voice) have been done before.
Why do people buy into this junk.
The easiest way is plain text. email/slack etc
VR is useless for the office. Things are solved via text/speech, occasionally a graph helps but we all know they are mostly for show.
A magical ability to make things 3D doesn't make work easier.
> In one of its demos the Magic Leap team shows off a computer-generated “virtual interactive human,” life-size and surprisingly realistic. Abovitz and his team imagine virtual people (or animals or anything else) as digital assistants–think Siri on steroids
How does a 3d Siri make the backend suddenly easier?
The AI to analysis what you are saying? The AI to answer correctly? The tech to speak smoothly to you? The things that actually matter and work fine in 2d.
A good computer game can costs 100's of millions. These are going to be even more expensive. Even gaming will take a while to kick off.
With investments from tech companies AND banks AND movie studios, and with half-a-dozen R&D outposts all over the world, it's hard to believe these investors just bought into vaporware.
I would expect that we've seen what MagicLeap is targeting in both Google Glass and later Microsoft HoloLens. So conceptually the systems and optics are proven at some early stage. Instead of projecting into a lens that is then reflected into the eye, they claim they are projecting directly into the eye. Either way the optics should be similar. This means that as with any AR (instead of VR) system, they will not be able to project the absence of light in a brightly lit environment. (Creating shadows/darkness should be impossible)
There are ways to hard-occlude, e.g. LCD mask, but might be punted til a later version.
Personally I am fascinated and optimistic about the potential represented by scanning fiber + waveguide display for AR and VR. I'm optimistic about the future of this tech platform far more than any other AR/VR tech out there mainly owing to form factor it affords (lightweight glasses) and the superior bio-compatibility of the light-field it produces (all day use). On principal its imagery is clearly light-years ahead of anything using a flat screen with lenses and its potential form factor could be surprisingly close to regular glasses. The image quality and form factor are huge part of why this tech could replace phone or a tv or a laptop. I am sure the first iterations will be cool but not perfect. I expect a progression much like that of the iPhone. It works, its awesome, then it keeps getting better and better and better.
IMHO this really does look like a viable platform for the future of computing for the next 10-20 years. People still buy PCs but the laptop largely replaces most uses for it other than hard-core gaming. Similarly I could see the same effect happening with screens vs ML over time. I say all this with the caveat that the form factor really is much much closer to glasses than something like HoloLense.
Here are couple things I have learned about the tech that kind of blew my mind:
* The same tech used to display the light-field can also be used to capture images of they eye for eye tracking, no extra camera needed. It is conceivable that the same scanning fiber technology could be used to dramatically miniaturize the sensors needed for SLAM inputs as well.
* The detail falloff of human vision from a small point in the center is huge. We largely piece together a detailed picture over time in our minds with quick eye movements. This means that good eye-tracking + foveated rendering largely mitigates concerns about achieving a very high perceived resolution with manageable IO.
* I assumed this kind of display would only be able to produce semitransparent ghostly images. Not so. Apparently it can block incoming light from points where it is projecting.
* It is not simple stereoscopy, the light hitting the eye is focusable at the distance the virtual object appears to exist in real space rather than at the surface of the waveguide. This is why it is so compatible with the human visual system and can be used all day as opposed to other tech. This has to do with how the visual system to point both eyes at an object in space is wired in sync with the system that focuses the lenses. No other system, to my knowledge, accommodates this linkage and thus causes strain. Watch the video to understand this more.
[1] http://uploadvr.com/magic-leap-how-it-works/ [2] https://talks.stanford.edu/brian-schowengerdt-human-sensory-...
there are also the deep technical problems which are widely acknowledged in the field and clearly not addressed. a top spec desktop machine struggles to keep up with your eyes movements /if they are known in advance/ - the idea that a pear of glasses coupled with /any/ peripheral, even one the size of your house, can contain the hardware to do this in real-time is quite far-fetched - even accounting for sensors with feedback and clever mechanical tricks.
there is no mention of how to deal with occlusion. no amount of throwing light at your retina can solve that problem... you need something to intelligently block it out. the marketing hype images make this seem like a solved problem...
maybe there is something that works, but i have no faith that magic leap is anything more than a scam targetting VCs. the same opinion i have grudgingly kept since i first saw anything of it... because its a cool idea and i want it.
i hope these issues are publicly addressed so i can buy it early in confidence instead of waiting to see if its as good as the sales-pitch... which, sadly, there is no indication it will be - because none of the obvious issues have been addressed.