Spatial – Collaborate from anywhere in AR
spatial.is
spatial.is
I think supporting cross "platform" is a really cool viral feature.
At first it's just going to be one guy leading the meeting with real full body tracking (think meetings with architects or mech engineers where you might want to discuss and move around 3d models)
Anyone else working remotely can facechat in with their phone (w/accelerometer & gyro or ARKit/ARCore) or skype in on desktop.
But every time they want to discuss something that's outside a non-vr/ar user's fixed field of view, the dude leading the meeting will have to rotate the model for them. They will feel left out and eventually will want to buy their own AR/VR setup.
(or more realistically they will preshare their cad files first ... but the above still sounds like a plausible future)
I'm curious about the hardware. Is there a base station or two hiding in a corner somewhere? If there is, why isn't head tracking accurate enough to prevent "sway" of virtual objects with respect to real-world ones, which seems to be visible in the video? But if not, how does it capture the position and pose of the user's hands? In any case, what kinds of sensors are being used?
Unfortunately, when watching the video, what really stands out is that the 3D "ghosts" of the other participants have juddery, unsmooth motion. Surely it couldn't hurt to add a bit of interpolation? It would increase latency, but not by much, given that a user's view of a different user's body pose is not especially latency sensitive.
Edit: On second look, it seems like the HMD is just a Magic Leap One, though that doesn't answer the question of whether there's other hardware in the room.
Hololens and Magic leap both have hand tracking, so the device knows if you are putting your hand up (and has some sense of its position and orientation), plus head pose.
"Sway" on device is actually more minimal that what you're seeing in the video.
How serious can this be taken given the crushing reviews of Magic Leap?
What is the rationale to make this happen in AR here? Is the physical location important for remote collaboration?
Additionally, we have all of these devices (laptops, phones) that we'll continue to use for the foreseeable future. VR forces you into a single mode where using your phone is impossible. AR lets you do more of a hybrid approach.
Long term I agree that VR will have more benefits (especially if you had hybrid ar/vr glasses), but in the near term it has too many drawbacks to be useful for collaborative productivity sessions like this.
If you absolutely need to drink water during a meeting you can just quickly move your headset up with one hand and drink a sip, while still hearing in your headphones what's going on.
You can do VR with a hybrid approach too. Same implications as calling in with a PC/mobile phone into an AR meeting.
Only upside I see with AR currently is:
- Easier to speak to people in the same room (but not necessary if half the people attending are remote anyways, in fact, it would level the playing field)
- Work on your computer during the meeting (which you shouldn't do anyways =P)
Standalone inside out headsets could work (quest), but people still would bump into each other since they wouldn't have a good sense of where other headsets are.
These drawbacks would be OK for some meetings, but dealbreakers for others.
In many of the meetings where you could remain seated, there's a requirement to screen share from a laptop (as you mention) that likely won't go away in the near term.
- Not sure what kind of meetings you attend where it's essential that people stand up. But I would estimate that for most meetings you can be seated (citation missing of course)
- With the Rift you can already access your desktop. It's only a matter of time until they launch that feature for the Go.
I think it will come down to how willing people are to not see the outside world, plus how tied people are to using (at least in my little bubble) mac laptops and google docs / web apps designed for laptops. I could imagine a mobile platform like quest or go having a web browser and a keyboard, but you'd have to have one of those logitech keyboards to even be able to see it to take notes.
Again to your point though, there are lots of meetings where you are just talking and where perhaps a simple mobile-vr-based web browser could suffice. For example, I'd potentially sometimes rather watch a movie with someone remote using a Go than using a hololens. There are also lots of examples of cases where everyone is remote and you want to control the environment that they come into (like a training session or school :) where it would probably make more sense too.
Re: Meeting types, the most valuable use cases for xr meetings right now are likely not 1:1 facetime style meetings, because it's difficult to replicate that personal connection in xr right now (video is better in some ways). One argument is that a more valuable use case would be brainstorming meetings (https://www.google.com/search?safe=off&rlz=1C5CHFA_enUS696US...) where people are often pretty active.
For brainstorming, just spawn a huge whiteboard in VR. It can be huge. With AR you're limited by the size of the room you're sitting in :-)
- Mischa @ spatial
With video, especially in a group, you can see if people are following what you say, nodding or shaking heads, have confused looks, or are not engaged (or falling asleep). Compared to voice-only discussions, for example, sometimes people won't ask the question if they're confused, presumably thinking they're the only one and don't want to waste others time and/or embarrass themselves.
Maybe facial expressions can be emulated -- but there's a very big uncanny valley to get over to make that usable.
a) meetings while taking notes or dealing with Trello and,
b) pair programming,
So, all I want is to stream my desktop and some aspects of my physical self (face and hand-movements are enough).There is no real need to bring in aspects of individual desktop apps into immersion (as they show with notes or 3D design tools), just stream my desktop and let everyone see. For pair programming, this would need to work seamlessly hours at a time.
P.S: Incidentally AR vs. VR is less of an issue here, however, if it was real MR (i.e. I can see my real laptop screen exactly where it is) and I don't have to stream my own laptop that would be great.
Things that are different irl vs the video:
1: Experience irl is way cooler. We use it internally for meetings. It’s amazing to take a photo on your phone and see it show up in the air in front if you. It’s awesome to see someone rez in and start walking around. It’s fun to put your hands up, start talking, and watch 3d models and images show up. The playfulness and interactivity is much more exciting in person than in the video.
2: The field of view on existing AR hardware is limited. Everyone working in AR hears this to the point of exhaustion, but anyone trying AR for the first time can’t help but notice it. So, the experience is different than in the video because you can only see AR through a smallish view. This will, of course, improve soon.
3: The video ui isn’t rendered additively and has some added zing that is less performant when using in real life (eg higher antialiasing)
A lot of the challenge in this sort of technical communication is conveying the Point of View of the person onsite. We have the technology now, so why not just render it?
Can you provide links? I'd like to see what they're up to.
I think you are slightly overestimating the capabilities of the technology,
No. I know how janky smartphone GPS is from personal experience, especially inside a structure. Big companies with deep pockets are working on Vernor Vinge's "localizers," however.
but largely overestimating the demand for this stuff over the way things are traditionally designed/built/inspected. It's not a killer use case that is forcing architecture/design/manufacturing firms to adopt the tech or die...it's currently still in gimmick stage, but is getting better slowly but surely
As you are implying, the demand is closely related to the jankyness event horizon. It's just like smartphones and tablets. They existed many years before the iPhone and iPad. However, prior to the iPhone and iPad, only propeller-heads wanted those things. There's a point at which the technology has matured to the point where it doesn't get in the way and it's actually nice to use. At that point, it will explode.
(and/or email the organizer to get slides/company names if you're not in boston)
Here's some AR companies working on architecture: https://www.archdaily.com/878408/the-top-5-virtual-reality-a...
One of the apps mentioned there, Pair, is run by Andrew Kemendo who is an active participant on HN, started his first AR company in the space in 2011 and is very knowledgeable, check his stuff out: https://news.ycombinator.com/user?id=AndrewKemendo
In the engineering space PTC/Microsoft (Vuforia/Hololens) and UpSkill (founded 2010) jump out in my mind as leaders. PTC has been doing CAD modeling and full PML/ALM tooling etc. forever and started to look at AR applications in the 2013/2014 timeframe to build on their modeling, maintenance, and workflow tracking expertise
A common thread is that many of these companies were founded in 2010/2011 and have rebranded around the 2015 timeframe to focus on different business problems and market their solution as "AR"
And yeah I agree, I think the jankiness of the hardware is still a huge sticking point, nobody has nailed the comfortable glasses form factor with enough POV/battery/processing power to really make AR take off
I get the AR piece: People who are physically in a location they see their remote peers magically floating in the room with them while they can interact with virtual objects. That's cool.
But what do the remote people see? They don't have the benefit of AR, so I'd imagine they don't experience the real-world environment as they're sitting at a desk with a headset on, or just simply viewing a 3D world like they're playing The Sims?
It feels to me like the "host" participants can use AR but anyone remote would need to be using VR as the physical environment mapping wouldn't make any sense for them.
There are some gnarly issues around this, of course, but Spatial has a pretty advanced approach that works in most cases.
https://spatialsys.github.io/res/shots/webapp/
https://spatialsys.github.io/res/shots/webapp/room_mars_terr...