A demo that Magic Leap was going to show at TED | Hacker News Reader

A demo that Magic Leap was going to show at TED

fastcompany.com

39 pointsmmastrac44 comments

This is unbelievable. As in, I literally can't believe it. There's no way they've come up with a full-resolution, head-mounted, transparent, additive and multiplicative display attached to a computer vision system capable of identifying light sources and nearby geometry that also has 100% perfect hand tracking with haptic feedback. I also don't believe anyone would make such bizarre, quasi-pornographic grunting noises while wearing such a device and "playing" at the office.

I must have missed the perfect hand tracking. Could you share where it was? All I saw was rather exaggerated movements to make things work.

This requires perfect hand tracking: https://i.imgur.com/QzSD0bq.png

I think the gun is a real, physical object -- I mean, it even casts shadows on the user's arm. It's probably just the enemies, HUD and glowy, flashy special effects that are rendered in VR, and those should be more forgiving of slight positional inaccuracies.

Especially as it comes from Weta workshop who originally designed the weapon (https://wiki.teamfortress.com/wiki/Righteous_Bison) - if anyone has a physical copy it's them

WETA have been selling physical replicas of these weapons for years: https://www.wetanz.com/rayguns/

Grabbing and handling the gun.

The gun is a physical object.

The interface widgets appear to be occluded by the user's hand, with no visible artifacts even when the hand is moving quickly. That means the system has to be tracking the hand and somehow separating it from the background, with latency of much less than a single frame.

EDIT: here's a particularly blatant example, around 0:17: http://i.imgur.com/8mSv8oC.jpg

The user is swiping from right to left, causing their hand to be highly motion-blurred. But the rendered UI elements smoothly fade in behind the hand.

There are foreground segmentation algorithms that can do this kind of thing, assuming you have some information about which regions of the image are foreground/background (say, from a depth sensor). But those algorithms typically take on the order of seconds or minutes to process a single frame. For this video to be real, the total latency -- sensing, matte generation, and rendering -- would have to be in the single-digit milliseconds. It's barely on the edge of plausibility, but I'm very very skeptical.

OK, that's right. I'm still not convinced anyone can say that this must be faked. None of this is impossible with current tech, just not likely in a wearable/battery-powered device.

Edit: Actually, no, I don't think it's right. Modern chroma keying could probably handle this.

If you had head mounted cameras, they could feed depth info to layer the UI, as you say. And if the display is covering your eyes completely, then feeding video in...? We don't get to see what the guy looks like, just what he's seeing. Maybe he's got 15lbs of Heath Robinson equipment on his head.

The devil is always in the details; or lack of attention to. Just slow it down on youtube to 0.25. If it wasn't clear before, the animations preceding the hand gestures will make it perfectly obvious this is a mock. Whoever they contracted the shots out to did not do a very thorough job of it.

Perhaps it's a head mounted display and all that geometry, the desks and walls and stuff on the desks, is built into the game level. If they really have a holographic display that's super thin, it seems like the rest could be hacked around.

As far as the grunting, how is it any different than having a Nerf gunfight or pingpong or whatever people are into at the office?

It's a concept video done by a special effects company well known for The Lord of the Rings and Avatar. The YT description "This is a game we’re playing around the office right now" is a bit provocative and without a real demonstration it's just PR.

Video on YT: https://www.youtube.com/watch?v=kPMHcanq0xM

Company: http://en.wikipedia.org/wiki/Weta_Workshop , http://en.wikipedia.org/wiki/Weta_Digital

It's called augmented reality and more basic implementations are already possible for some time: http://en.wikipedia.org/wiki/Augmented_reality

@Downvoter: what's wrong?

I didn't downvote you, but I suspect it's because this whole comment thread is about trying to determine whether or not the video is real footage of Magic Leap's tech. Asserting that it is or it isn't, without something to back up that assertion, isn't all that helpful. Nowhere in the video description does it say that it's just a concept.

> without something to back up that assertion, isn't all that helpful

The video credits list Weta prominently - that's alone is a significant signal. The guns are props from Weta too: https://www.wetanz.com/rayguns/

CEO Rony Abovitz, who was supposed to go onstage at the TED conference in Vancouver on Wednesday, has canceled his talk. Why would a CEO cancel such a high profile talk and release that video afterwards? More info: http://recode.net/2015/03/16/magic-leap-googles-high-profile... , http://recode.net/2015/03/19/the-first-video-of-magic-leaps-... .

I tested two state of the art augmented reality glasses (university research prototypes) last year. The glass was connected (via cable) to a workstation PC with several high end NVidia graphic cards. One also used Xbox Kinect to track the environment and the arms. Everything shown in the video is already possible, but not with that perfect visual quality and without any latency.

> It's a concept video done by a special effects company well known for The Lord of the Rings and Avatar. The YT description "This is a game we’re playing around the office right now" is a bit provocative and without a real demonstration it's just PR.

It's worth noting Peter Jackson has made an entire documentary on a forgotten genius of the New Zealand film industry that turned out to be a (good-natured) hoax. I'm perfectly prepared to believe the guy responsible for Forgotten Silver would have a team around him who might do the same for VR.

There is a markedly difference in resolution between the real world and the AR stuff.

Especially when the actor scrolls through his mail, his hand is a very low resolution and the mails are a very high resolution.

I suspect this is a major signal of it being fake.

Wait, I'm confused. Is that video the actual game and how everything actually looks, or is it a concept video with visual effects and props and whatnot by Weta? Because up until now, I was under the impression that it was the latter.

If that footage is in-engine I'll eat my hat.

From their official YouTube account: https://www.youtube.com/watch?v=kPMHcanq0xM

"This is a game we’re playing around the office right now" is extremely provocative.

Edit: ah, you removed your request for citation. Doh. :)

Sorry for mixing things up with a ninja edit!

The article says:

> This is the video of a real-world, first-person shooting game that Magic Leap says it was going to show at TED this week.

At first I parsed that as saying Magic Leap was going to show the game at TED, but what I've seen other articles say is that they were going to show this video at TED.

There is no way this is in-engine AR. This has to be a concept video for a prototype they're working on.

For them to say they're playing this around the office isn't exactly a lie -- they may very well be working on a prototype that has similar mechanics -- but it's bending the truth.

I'm not clear what's so surprising to you. AR has been progressing rapidly. This looks like the next iteration. It's not some giant leap from what we've seen before, or even what's in millions of consumers' hands right now.

No it's really not. This is a long way from anything that's been shown so far and is clearly a mock-up. The devil is in the detail, the production value is too high and the visual integration between the CGI and real elements is too slick.

http://13thlab.com who got acquired by Oculus made something very very similar:

https://www.youtube.com/watch?v=rA2NI4NgsV0&t=19

so the technology for doing this is feasible and real.

Their previous kickstarter: https://www.kickstarter.com/projects/1203948132/rescape-fps-...

That's a cool-looking video. It looks like it isn't AR though -- it looks like they're taking the real-world position of the phones and using that to drive the positions of the players in the game. They've modeled the game so the level looks the same as the office they're playing it in.

Magic Leap PR has not yet confirmed if the video is an authentic demo of the product ( http://recode.net/2015/03/19/the-first-video-of-magic-leaps-... ).

One hypothesis is that PR commissioned the video and planned to show it at TED and other venues. IMO the video definitely looks produced and it was prudent to cancel the talks rather than spin the video as a plausible demo of their product.

Re/code's comparison to Google's hype of Glass is a little harsh, but nevertheless if the video is indeed "fake" this would be yet another example of Marketing being completely reckless with consumer expectations. I'm sure the Magic Leap product has some impressive strengths. Better to let the product speak than to let Marketing try to "lift" something different that doesn't exist.

Re-watching the glass video, compared what was actually launched, their criticism seems fair in retrospect.

It is also a great comparison on another level. This product may be amazing, but whatever it takes to make it happen is so unwieldy that no one would want to use it. Glass didn't fail only on the product experience, but on the social stigma aspects as well. Odd gruntings aside, until we actually see the product it is hard to say it will sell.

A good example of this is Apple. Presentation is equally important as the actual tech (hence these unrealistic marketing videos.)

It's odd that Engadget trumped this. I'll re-hash my comment from the article:

Those weapons are physical props from the Weta workshop collection (starting at the low low price of ~$500, up to about $7000 @ https://www.wetanz.com/rayguns/). They also used SFX from Team Fortress 2 when they put down the turret. For the video they either:

a) Created a magical bio-feedback system to artificially create gun recoil for the person holding the weapon as well as tactile feedback for "in the air" functions.

or most likely:

b) Created a Post-processed CGI demo meant to impress execs that have no concept of what's currently possible - Might as well hire CorridorDigital and make something like this instead: https://www.youtube.com/watch?v=OhCQeFX9GSg

I don't even understand the purpose of a video like this. The only thing it tells me is what Magic Leap hopes their tech will some day achieve.

Blatantly conceptual and given that it's being released in lieu of in-person presentations, one can only assume that they didn't want to have to publicly explain it.

I'm not sure this is the actual thing. If you check out the video at near 0:10 you have emails which are fictitious, like an email from brands.edu (no whois record).

For the company to say they're playing this at the moment in office it is a bit misleading.

I hope the real thing will be just as cool if not cooler.

I'm skeptical about the technology, but the email domains have nothing to do with it. They could easily create a demo with real AR hardware and fake data, or with fake visual effects and real data.

The software is pretty clearly a mockup, but nobody cares because the 3D interface design isn't all that interesting. It's the hardware that's claimed to be revolutionary.

The fact that the guns have recoil seem to suggest it's a concept video.

can't believe anyone thinks this is real. It might resemble where they want to go with the product but this video is composited, that's blatantly obvious. I hope they can deliver on this vision. cough google glass cough

Yes.

Let's not forget this is the same company whose patent draft diagrams were found to be basically wholly ripped-off copies of other public works. [1]

I'm impressed with the thoroughness of HNers screenshotting and looking at motion blur, but nothing I've seen says we should give this company the benefit of doubt. This reeks of "concept video" all around.

[1] http://www.theverge.com/2015/1/30/7954611/magic-leap-augment...

Of all the skepticism, the two most reasonable ones are - quality of the headmounted transparent display. This doesn't even seem to be Magic Leap's core competency and the tech required for a HMD to reach this level of quality is far beyond anything we've seen before. - real-time computation power on a battery powered wearable. It would run incredibly hot and drain so much power unless the guy was wearing a giant battery/CPU backpack covered in exhaust fans.

For the other things, 1. The room geometry could have been prescanned and coded in 2. hand-tracking could have used an additional wearable sensor on the hand 3. the AR systems could have been optimized for recognizing the gun (I have worked with systems capable of that) for highly accurate and fast real-time tracking of a non-malleable object 4. and the interactive 3D graphics given knowledge of the room (surface geometry and lighting) is not a big deal at all.

The acting (such as recoil on the gun) is a bit overdone and obviously has nothing to do with the technology itself, and I find it weird that the AR actually seems to assume you'd recoil that much and animates the shots as such.

> The room geometry could have been prescanned and coded

There's an Oculus Rift forum post that has a link to hand tracking with the Structure Sensor.

The depth sensing portable Structure Sensor is 95 grams and $380 (Kickstarter).

There's noticeable latency in the hand tracking, but a post talks about possible low latency for static objects by using a previous map.

Assuming you stay within a certain space, would premapping with something like the Structure Sensor or Project Tango contribute the most to the stability of virtual objects that Hololens testers experienced? (objects stick to their intended position as you move):

>Daily Tech News Show DTNS 2414 - Jan. 23, 2015

that objects stayed stable when you move your head 17:43

well hold on a sec it is excellent 17:47

at keeping orientation for sure I looked around and something with an absurd and 17:51

Spa 17:52

I looked around the same thing was there again I if they're mob rock on the 17:55

surface of mars 17:56

or a table covered with you know i i 17:59

toy castle at minecraft made all these tiny little minecraft building blocks 18:03

it would always be exactly where I thought it would be but I had to turn my 18:07

head all over the place cuz the field if the US 18:10

tiny it's like this little I somebody's described it as a sixteen by nine TP 18:15

floating maybe seven to eight feet and Franny 18:18

you're looking through this little narrow slice ever went out about this 18:22

big around in my view 18:24

trying to see mars this much at a time 18:27

and wherever you look is like oh that's exactly where I thought it would be 18:30

but its tunnel vision it's like you're looking through a pair of binoculars or

watch?v=uGYksMRmUwA

---

>Through it all, the 3D effect was thoroughly convincing.

>The system felt very low latency; as I moved my head and walked around, the objects retained their positioning in the real world, with the castle, for example, never becoming detached from or wobbling around on the table.

arstechnica

---

Or is good real-time tracking necessary for the virtual objects to stay attached to their positions.

(> If you can figure out these numbers for your system, the addition of 38 ms from the Structure Sensor will give you and idea of the total latency in your system. kickstarter

> 20 ms minimum latency (AKA lag, compared to 102 ms minimum latency for Kinect 1 and 50 ms minimum latency for joypads in 60 fps games.) 123kinect)

Reality Check: Comparing HoloLens and Magic Leap: http://www.technologyreview.com/news/535806/reality-check-co...

They need some UI & UX designers badly.

Although I used to enjoy quake2 when I was young, I don't feel the appeal of shooting games anymore, or videogames in general. That is to say, I have been on both sides of the fence and I still vividly remember the sweaty excitement of a deathmatch.

Now, my first impression to this and to other AR (and VR) demos is, "Isn't this literally a new Columbine waiting to happen?".

(NOTE: I do not mean the incident, those seem to happen regardless, I mean the public outcry and witch-hunting).

Has public perception of interactive violence shifted to a point where "it's not different from an action movie after all", even at this new point of immersive realism?

I think anyone who has issues distinguishing between virtual reality and the real world will have issues regardless of whether or not they play these types of games. I don't think there is any study out there showing that violent video games make people any more violent.

And I agree with you! Edit to make the point clearer.

Reply on news.ycombinator.com