Daydream Is Google’s Android-Powered VR Platform
theverge.com
theverge.com
I fear the fracturing will make total adoption lower and slower, as developers will have to choose sides or spend way more time developing for all platforms.
You'd think we would have learned this by now, but I'm still way too excited about the vive.
The whole point is to use Unity's single high level device independent VR SDK (which enables many internal rendering optimizations), and to avoid plugging in yet another VR SDK in addition to a bunch of other separate GearVR, Oculus, Sony, etc SDKs, and having to write complex branching code (and build configuration scripts) in my app to support multiple APIs with my own ad-hoc layers of abstraction (which is costly to maintain and soon to become obsolete). That's Unity's job, and why I gladly paid them for a license.
Unity has a device independent VR API that supports Oculus, GearVR, Sony and other devices, but as far as I know it does not yet currently support Google's Cardboard SDK. They've promised to support it, but they haven't delivered it yet, nor said when to expect it.
Unless Google wants to pay me for wasting my time, the last thing in the world I want to do is to spend a lot of time and effort duct taping Google's Cardboard SDK onto the side of my existing app that already uses Unity's built-in VR SDK, and then Unity finally releases their the built-in support for Google Cardboard that they've been promising.
The potential and promise is there, but until I can just tick the "Virtual Reality Supported" checkbox in the Unity player settings to transparently support Google Cardboard, GearVR, Oculus, Sony, Vive, etc, the rubber hasn't hit the road.
Until then, you have to do short term dead end hacks like this, which will probably break every time anything it depends on releases a new version (which is often): https://github.com/ludo6577/VrMultiplatform
Also, even if they do automate handling the rendering and input aspects of each VR SDK, you will still have to deal with the various different platform / store level aspects. Google Play versus Oculus Home versus Steam versus whatever else comes in. It's already a struggle maintaining separate Rift/GearVR builds in Unity, when I have to maintain 4 different environments and their platforms? Yeah...
They're the only ones in the position to really solve the problem at the right level, since it interacts so deeply with their rendering and input system, which is all under the hood so not possible for external developers modify, hook into, or fix problems.
Unity needs to integrate the input system with their new UGUI user interface toolkit, and integrate it with VR devices so all the different kinds of input devices plug in and are handled consistently. So they need to get very friendly and cooperative with all the different VR and input device hardware manufacturers they support, and do whatever it takes including flattering, shaming or partying them into cooperating if they foolishly decide they'd rather lock their developers into writing code specifically for their device, so that it doesn't work with other manufacturer's devices on purpose. (Here's looking at you, Apple!)
Unity has announced a new input system [1] and published a prototype [2], and asked for developers to give them feedback [3], but it's way too early to use in a product.
I'm optimistic that they mean what they say, and glad they're publishing the prototype and asking for feedback from developers, but I appreciate it's a difficult problem that will take a while to get right.
I've read over the documentation, and it seems to take a nice modular approach that can abstract away many differences between input devices, but it's just a hard problem by its very nature, and there will always be special circumstances for particular input devices that you need to handle on a case-by-case basis. So both the VR API and the input API should support hooks and customizations and plug-ins for special purpose hardware, and ways of querying capabilities (like position tracking), enumerating devices, reflecting on and hooking into the model.
I'm disappointed that neither the new UGUI user interface system nor the new input system prototype seem to fully support multitouch input and gestures in a nice way, like the TouchScript library does [4], but I hope they eventually support that as well.
One frustrating example of a bad API that both the Oculus and Unity SDKs have is the "recenter" call [5], which resets some hidden state inside the tracking code so the current direction of your head is "forward". You can't read and write the current yaw nulling offset [6], you can just "recenter", whatever that means. So there's no way to smoothly animate to recenter -- it always jerks your head around. They should expose that hidden state, and let me read and write it, so I can recenter smoothly. It's never a good idea to have a bunch of magic and hidden state behind an API like that. Plus the documentation is terrible -- they don't define what any of the terms mean, what the model is, or what the actual effect is mathematically. (Does it reset the yaw around the neck, resulting in a discontinuous jump in eye position? How can I tell what those measurements are, or know what the model really is? Does recenter work differently with devices that track the actual absolute position of your head, as opposed to estimating it from a model of the eye position relative to the neck?)
The input system should also support other kinds of gesture recognition like motion tracking (staring, tilting, pecking, shaking and nodding your head), popular "quantified self" input devices like the ShakeWeight with its optional heart rate monitor [7], network and virtual device adaptation, emulating and debugging hardware devices in the editor, etc. But right now it's so early in the game that the new input system prototype isn't yet integrated with the new VR API or the (less) new UGUI toolkit.
Oculus has examples of how to integrate head pointed ray casting with UGUI using reticles and cursors, but that code is brittle and dependent on their particular SDK and app-specific utilities, and it copies and modifies a bunch of the Unity UGUI input tracking code instead of subclassing and hooking into it, because Unity didn't expose the public virtual methods and hooks that they needed. They've beseeched Unity to fix that by making their API more public and hookier, but for now, Oculus's example UGUI integration code is pretty hacky, complex, and not a long term solution.
All the input tracking stuff will work much better when Unity fixes it under the hood (and lets developers open up the hood and hot-rod the fuck out of it, or shop for competing solutions on the asset store), instead of having 6 different VR and multitouch SDKs that all solve it in 6 different but overlapping ways, which you can't integrate together.
[1] http://blogs.unity3d.com/2016/04/12/developing-the-new-input...
[2] https://sites.google.com/a/unity3d.com/unity-input-advisory-...
[3] http://forum.unity3d.com/threads/welcome-new-input-system-re...
[4] http://touchscript.github.io/
[5] http://docs.unity3d.com/ScriptReference/VR.InputTracking.Rec...
[6] see "nulling problem" -- this is a GREAT paper: http://www.billbuxton.com/lexical.html
Lexical and Pragmatic Considerations of Input Structures
http://www.billbuxton.com/lexical.html
PRAGMATICS & DEVICE INDEPENDENCE
"From the application programmer's perspective, this is a valuable feature. However, for the purposes of specifying systems from the user's point of view, these abstractions are of very limited benefit. As Baecker (1980b) has pointed out, the effectiveness of a particular user interface is often due to the use of a particular device, and that effectiveness will be lost if that device were replaced by some other of the same logical class. For example, we have a system (Fedorkow, Buxton & Smith, 1978) whose interface depends on the simultaneous manipulation of four joysticks. Now in spite of tablets and joysticks both being "locator" devices, it is clear that they are not interchangeable in this situation. We cannot simultaneously manipulate four tablets. Thus, for the full potential of device independence to be realized, such pragmatic considerations must be incorporated into our overall specification model so that appropriate equivalencies can be determined in a methodological way. (That is, in specifying a generic device, we must also include the required pragmatic attributes. But to do so, we must develop a taxonomy of such attributes, just as we have developed a taxonomy of virtual devices.)"
Also check out Proton, which is a brilliant regular expression based multi-touch gesture tracking system, which would work very nicely for VR and multi-device applications. Proton is to traditional ad-hoc gesture tracking as Relax/NG is to XML Schema.
Anybody buying now has to recognize that anything they're buying will be obsolete in a year or two. Once you realize that, you realize it doesn't matter too much if you pick the 'wrong' one. You'll be replacing your headset with v2 or v3 of the winner anyways.
Early in the lifecycle, when there is a lot of R&D overhead and risk, is where companies actually have incentive to collaborate. It's very unlikely that the ecosystem will "open" later, instead we'll have to (as you said) accept the ecosystem of whoever wins.
It's all a part of the race to the bottom, from cloud data services to internet browsers.
One major problem is that none of them are open, and none of them have pushed to create standards; they all want to unilaterally define themselves as the standard. I think we'll get a more unified ecosystem when someone starts looking to collaborate on standards rather than exclusivity. Perhaps Google will do that with Daydream, or perhaps someone else will.
High End - Desktop ( Vive / Oculus ) $600-800
Console ( PSVR ) $400-500
Mobile ( Google / Oculus / ?? ) $99-??
Console vs Desktop vs Mobile -- that's the real issue. The leap from Mobile VR to Desktop VR is huge. PSVR -- remains to be seen what Sony does on the hardware front (PS4.5 with new ATI card, or ??).
After all, to this day gaming is distributed already between console/desktop/mobile, with each audience being large enough to ensure good profitability in every environment.
No, you won't have literal one-to-one code, at least outside of WebVR. But I highly doubt porting is going to be that hard.
If anything, because you have to do the graphics on your own, there is an even greater chance of successfully building cross-platform systems. At the end of the day, it's all just GLSL.
One platform to rule them all will come in time, but we should be happy that for now the ecosystem is competitive, it means the consumer gets a voice in deciding what shape the tech will take
New technologies always results in lots of competing platforms, then consolidation, then stagnation, then a new competitor entering, new features, market chaos, then consolidation again.
And people will complain at ever stage.
And it has always been thus.
Certainly if different systems support different amounts of a standard (eg browsers and the web standard), fractured seems like the right term.
Is the requirement for a fractured ecosystem just different APIs for the same thing? Are ML frameworks fractured? I suppose they would be.
What's the solution? Would you propose a VR standard? I imagine cardboard was an attempt at that. But standards are slow moving - not very effective at innovating.
I think a standard will naturally emerge once we know what should be standardized. But you don't want to kill innovation yet.
Does that make total adoption lower and slower?
My bet is that in the future, VR will become more and more similar to what the Vive is offering now. Room-scale VR is a game changer, as much as VR itself.
I'm waiting for reviews and availability of the GTX 1070, as well as better headset availability before pulling the trigger on a headset. Hopefully Oculus has their controllers out by then, otherwise I probably won't wait for them. (I won't wait for AMD either; if they want me to consider Polaris, they better release it ASAP).
They haven't even shipped all their headset preorders yet, I doubt "soon" is the word you want there.
[1] https://play.google.com/store/apps/details?id=com.google.and...
In the latest Android N preview, it's been renamed to simply "Screen saver".
Google Bob! ;)
Positional tracking is a really challenging problem with a lot of limitations. If they were using computer vision to track the controller it would need to be held within the device's FOV and have identifiable elements such as a marker or LEDs. There are other approaches, but nothing I've seen that would work well on an HMD.
I've been working with AR/VR for the better part of a decade, and controller tracking's come up several times. I can tell from my own work that the device they're showing off is IMU only. It's too bad, something equivalent to the Vive controllers for mobile VR would be fantastic.
And at the end of the day - you're right. If it was spatially tracked then they definitely would have mentioned it.
First, I think positional tracking of the person within his/her environment is one of the main things that makes the difference between VR and AR. So there's value in that. If you can do it with computer vision maybe it can work in almost any environment.
Second, there are RF based positional tracking systems. They're more expensive, true (not if you can design chips yourself, so might even be cheaper for google), but you don't need the controller to have visible elements.
Third, hand tracking like this should be able to work usefully, shouldn't it ? I know it doesn't do distance, but ... Just hang it off the viewer pointing downwards with a large volume immediately in front of the user. Not great for gaming probably, but for a virtual keyboard it should work, no ? http://www.ctxtechnologies.com/products/vk-200-keyfob-virtua...
There'd need to be either:
- a very expensive IMU in the controller, talking $300-500
- a camera and hefty CPU in the controller,
- a camera and dedicated image processing CPU on the headset
To be able to integrate the acceleration data from an IMU to get positional data out of it is not something you can do with commodity IMUs, even the one made for Oculus. They're too imprecise and too inaccurate and too slow, and any analysis tricks to improve the results all introduce latency. Oculus' sensor has very little drift (though it still has drift, which is another thing the camera system on the desktop Rift can correct for) and the noise in the orientation data is not noticeable or can be filtered much more simply without introducing "too much" latency.
There are IMUs that are good enough to integrate acceleration data to position data, but they are prohibitively expensive for this application.
For what it's worth, I used to see a marked difference in the quality of orientation tracking between my Galaxy Note 4 in Google Cardboard vs. the Gear VR, but I see no different between GC and GVR on my Galaxy S7.
My prediction is that this will play a huge role in mass adoption of VR!
I'm going to start researching this platform now and will invest my time as a developer in it as soon as possible. I predict that this will be a good platform to get behind. Perhaps I'm being naive, but in a space with as much market potential as VR, this vision has me incredibly excited.
edit: One more thought: The hardware that they showed in the demo appears to be a step ahead of GearVR in usability as well. The headstrap isn't painfully assembled (my GearVR unit falls apart half the time I put it on) and it comes with input controllers. GearVR is woefully lacking in input options out of the box.
http://blog.imgtec.com/powervr/presence-in-virtual-reality-a...
ALSA isn't flexible enough, PulseAudio is too slow, JACK is too specialised, etc... CoreAudio on OSX shows what an audio stack can be capable of, all the Linux audio solutions currently available are subpar compared to CoreAudio.
AudioFlinger is Android's audio server. I'd be interested to see how far this can be pushed, but perhaps Google will route around it and provide a different solution for the type of low latency audio that immersive VR requires, IIRC using an alternative audio stack was Samsung's approach with GearVR.
I have a Vive at home and its wonderful for gaming, if a bit undercook and still unable to deliver a pixel density that makes me happy (Vive2 perhaps?), but I can't imagine a phone remotely competing with that still underpowered experience.
I can see AR projection built into one's existing glasses a la Google Glass or perhaps what MS is doing with AR, but VR is a totally different beast. Comfort, performance, fov, pixel density, graphics quality, audio quality, "presence," etc really matter. I just don't see Google pulling that off and even if they got close, who exactly is clamoring for VR phones? I suspect Google has just become too mobile centric and is shoehorning in whatever is hot into its Android line and seeing what sticks (instead of refining Android to be a better experience it seems). I'm not sure if that's wise. VR seems to be more at home attached to a powerful computer in a safe indoor space where people can feel free to move around without injury and get a high quality VR experience. You shouldn't be doing VR at the bus stop.
For example, what if every TED talk had a VR broadcast so you could slap on your goggles and "attend" it. This goes for conference key notes or college lectures. Anything where Physical attendance is a barrier and not a key component of the experience I think would benefit from a lightweight VR experience like this.
They're not ready yet:
http://techcrunch.com/2015/10/28/a-live-nba-game-is-cool-in-...
but it's definitely coming.
I think attaching a phone to your face is going to give a subpar experience compared to a dedicated device. I still don't see a use case here that's going to get people excited. Especially after the largely milquetoast reception smartwatches have gotten. VR masks are about 1,000x more geeky than those. Image conscious people aren't going to be putting them on their face to consume content in public.
Just minutes ago a co-worker grabbed the office Cardboard VR and slapped in his phone, and watched some of the Google I/O keynote. Cheap. Easy. Portable. Low res, but there he was virtually at the presentation.
People will get used to the oddity of VR just as they did with cell phones, hands free ear pieces, SIRI, coffee shop computing, etc and will get over self driving cars.
My Dodo Case Cardboard VR folds up and fits nicely in my messenger bag. And I like my Apple Watch, thank you very much.
Yours is a naysaying template I've heard for nearly every new technology for decades, and here we are with most all those things now ubiquitous.
iPods fit in pockets. VR headsets don't. Did you even bother to look at the Google reference design? Its a huge headset and controller. Those aren't convenient mobile things. They're not remotely portable like a phone or ipod is.
They still have a place without having to be tied to a computer. Like I said, we have a Cardboard at the office, same dimensions you object to, easy to have around, easy to share, cheap, no dedicated computer needed. There are more phones in the office capable of using the head mount than there are people. Need more head mounts? cheap order from Amazon, arrive in two days. No hassle of sharing a computer for it, just pop in your phone.
And like I said, some units do fold up convenient for travel.
Because a "real VR headset" includes the cost of the display screen and requires an external control device, whereas a mobile VR headset uses a mobile device that you would probably have already anyway as both the control device and display, and so is significantly less expensive and less hassle to use, even if you never use it anywhere but inside your own house.
> When the prices come down on this stuff it'll just be seen as another peripheral for your computer.
Mobile VR is just another peripheral for your computer -- the little computer most people carry around in their pocket, not the big clunky ones that less and less people are bothering with.
EDIT: By "control device", above, I mean the processing center, not the user interface device (which will still need to be separate for a mobile VR device.) I see that that could be misunderstood.
[1] Yes, I'm aware the computer market is huge, and there will continue to be a huge market for computer based VR and I'm completely wrong about this. BUT the mobile market is bigger.
No thanks. I currently would rather read a paper than youtube a lecture. I would surely rather read a paper than strap a $600 device to my head.
One of my favorite presenters, for example, is David Beazley who has several presentations/examinations on the Python GIL I found very useful. As another Python example, I also found Andrew Montalenti's overview of multi-processing in Python quite enjoyable as well (https://www.youtube.com/watch?v=gVBLF0ohcrE). Finally, Sal Khan was such an inspired presenter that he founded Khan Academy (https://www.khanacademy.org/) based on his great success teaching mathematics to his younger relatives and their classmates through YouTube videos.
This has utility for note taking as well.
I'm pretty sure I've watched corporate internal presentations with this style of UI -- it's quite useful.
Imagine walking through an AR doorway overlaid in your home, where you can see a beach on the other side. You walk through and magically are on said beach. You turn around and the door is gone--the illusion is complete.
More realistically, I see AR increasingly taking the form of your "main screen" with notifications, browser windows, etc. all accessible from there and just taking up visual space in your FOV. If you decide you want to focus on work or engage in another more immersive experience (movie watching, gaming, etc.), you slip into VR mode and suddenly the distractions are gone.
Again, lots needs to happen to get us there, but I see the blending of the devices as inevitable once the form factor is popularized. My guess is it will take something as sexy and revolutionary as the first iPhone to get this to go mainstream since you are right in that most people will not carry a giant headset around.
Yes. The games and immersive videos are cool, but I'm even more looking forward to replacing almost all dedicated displays with a pair of normal-looking glasses.
By now it should be way, way better.
Whoever is driving this shit off the cliff - please just stop while you're ahead, go back to designing hospital websites or whatever it was before you tried your hand at this, and let someone else do the whole newfangled "VR" thing instead.