The Road to Virtual Reality
codinghorror.com
codinghorror.com
http://vr.mkeblx.net/oculus-sim/
You can see there's a pretty huge difference between the 800p one and the 1440p one (~4x more pixels), which will probably be what the consumer version will have. With 1440p you will still notice the screen door, especially in the distance (the sky), and it looks like we'll only solve this with the 4k resolution, when the resolution seems to become "normal", and without easily noticeable pixel lines, but it will probably be at 8k when we'll have "retina"-like quality.
All of that being said, I think 95+ percent of consumers buying the 1440p version won't even notice the small pixel lines in the distance, because they'll be so "present" in those VR worlds. Most "normal" people trying it out quickly forget about the screen door even with the 800p developer kit, because they've never experienced something in VR before, and to them it feels very real already. Personally, I do hope they can use the 4k resolution in the second consumer version as soon as possible, but I think the 1440p one will do just fine with all the early adopters.
I'm hopeful 1440p will be fairly text friendly and prove good-enough for programming, etc. but even if not quite there, that merely means that threshold will be broken a year or two down the line.
The problem? It was cost. We had a tentative agreement to put VR racing pods in a Las Vegas hotel, but after the prototypes where developed (and they were great fun!), we did better cost analysis and cancelled the project because the "ride" would have been way too expensive for end users. The custom motion platform was especially expensive, but added so much to the experience.
Later at a different company, I was lead programmer for a VR demo system for Disney. The demo was great, but when they deployed to production in Disney World they had to cut corners on the motion platform, etc.
edit: my point is that there is a lot more to VR than just 3D graphics - if you want to achieve what is called "suspension of disbelief."
Based on the reviews I've seen of the Crystal Cove prototype, it looks like this new 1:1 head tracking, coupled with reduced motion blur, has eliminated a majority of the nausea from the experience (1).
So if Oculus holds up their end of the bargain, the next breakthrough is on the game developers. Who is going to create that 'killer app'? My bet is on Valve with something like Half Life 3 / LFD2 (2).
1 - http://arstechnica.com/gaming/2014/01/new-oculus-prototype-f... 2 - http://www.escapistmagazine.com/news/view/131719-Leaked-Sour...
If we're talking mainstream breakthrough.... If EA weren't so backwards, and assuming the hardware were ready, The Sims VR would cause an earthquake.
Also, a next generation incarnation of Minecraft would be a prime candidate. A world you can 'live' in, that is easy to randomize and wouldn't take five years for devs to build first (ala Skyrim).
The only cause of unbearable motion sickness for me was strafing in games like HL2, Portal, etc. I'm somewhat skeptical that adding head tracking will make these strafing-heavy games sickness-free.
Currently, the sickness is such an off-putter that I need to force myself to keep using the device. I'm somewhat concerned that if this cannot be eliminated for all content, then some proportion of users will still get sick as the envelope is pushed. I know that for many/most consumers, even slight sickness will cause them not to want to use it at all.
Many of you are caught up in the small technical details. Forget about that and look at the big picture. We are finally going to replicate experiences and our brains will really think we're there.
This will have repercussions on things that may not be so obvious right now. Things such as traveling for meetings or entertainment. Why go through the hassle of getting ready, driving through traffic, looking for parking, etc when you can just put on VR equipment and experience the same thing instantly?
I can't say for sure that VR will be as big as I think it will be, but I'm pretty confident that it's an either/or thing. Either it's going to be the most impactful technology since the internet (perhaps greater), or it's going to be a flop. I don't think there will be a middle ground.
That seemed really obvious to me in the early 90s, too. I don't mean that to sound dismissive or get-off-my-lawn-ish, just to point out that there's a history of VR overhype which makes skepticism understandable.
Now though, we're at one of those Singularity-like points of convergence which makes the plausability of mass usage of this kind of tech quite conceivable.
We have the resolution, the processing power, apparently the response times and positional-tracking too. None of these were practicable back in the 90s.
I, for one, am getting a wee damp at the whole idea.
To think that now I could probably purchase an Oculus and code that myself. Makes me excited to see where we are going from here.
Over the holidays, my brother-in-law brought over a set of the current Oculus. The 720p ones.
The experience reminded me a lot of the early days of 3D, i.e. the difficulty alone of setting up a 3D daughter card properly. You needed the right version of this driver, the right version of that app. The Oculus took about 45 minutes to set up properly - and I am a gamer with a gamer PC. We needed a particular game from Steam that had a development feature that you could turn on, etc. We had to fool around with the Windows video settings. You name it. This will obviously get better when they release a real version, but it was pretty bad.
Then I actually tried them. All of Jeff's observations are spot on. The current resolution sucks. The problem is that your eyes are so, so close to the screen. I thought that I was going to have problems as a user of reading glasses, but that turned out to not be a problem. I have no idea why, but I didn't need them, so that was good.
The other big issue is maybe health related. These things can make you (or at least some people) sick as a dog without a good deal of practice. I'm not sure if it's the lag, or whether it is a fundamental property of having your brain yanked from the visual environment that it evolved in for the last 100+ million years. It was really bad. Of course, the game we were playing, which was some sort of flight simulator, didn't help. I did not try another game. I didn't feel like I was going to be able to walk after 10 minutes of VR.
One other thing. These goggles invite new gameplay experiences that probably haven't been invented yet. And by that, I mean that just strapping these things on does not suddenly make current mediocre games great. Mediocre games still suck. In fact, they are worse, because you can see every flaw, due to the closeness of the screen. So there will need to be games that are made just for this device. Which, I guess means that you should not buy these hoping that your current set of games will now become "walk thru" or whatever.
It is going to take a compelling combination of hardware and software in order to make this device really shine.
I wish them the best. I am really on the fence as to whether or not this is a revolution at all. Time will tell. I hope it is and that they can overcome all of the hurdles, many of which seem kinda big.
It's primarily a matching failure between what you see and what your inner ear, proprioception, and sense of pressure are telling you is happening, as well as any other visual input if it's a mixed-vision system. Mostly inner ear, and vision if present.
Also it's mostly rotational motion that triggers it - but you'll easily hit that with just the small motions of your head and the looseness of the headset.
Which is ridiculously hard to compensate for since you have to get extremely precise, low-noise motion & environmental pose tracking, with extremely tight time limits - and then it has to be fed into the rendering engine, which has to react in time to fool the operator.
But the evolutionary rationale is also right. Your brain evolved to freak out in the event of persistent sensory mis-match because it's a pro-survival feature. The naturally occurring causes are things like severe food poisoning.
> I thought that I was going to have problems as a user of reading glasses, but that turned out to not be a problem.
IIRC, the reason is that VR stimuli are "focused at infinity." It also helps that you're trying to take in the image, not the pixels or other fine details.
Trying to read small text at a distance of ~ 20 cm requires relatively precise focus. A picture of a cat that's taking up your entire field of view does not.
I just read the Abrash PDFs and they are very honest and insightful. It's nice to know that someone (many someones!) is actually doing the science that's needed to get this stuff right.
Honestly, given the challenges, it seems reasonable that this will all start the way that the 3D game revolution did. Slowly, on the PC first. One step at a time.
I'm in as soon as there is a compelling game to play!
Here's a little resolution/density chart I made that might be useful: https://docs.google.com/spreadsheet/pub?key=0AgAh7-k-pFfVdFZ...
Farsightedness is fine - the Rift focuses at infinity. It's probably a lot healthier than staring at close screens in terms of eye stress/focusing.
Simulator sickness is mostly caused by excess latency and confusion of the vestibular system. The latest word is that these are taken care of with 1) accurate positional tracking (DK1 only has rotational tracking), 2) sub-20ms motion-to-photon display, and 3) low persistence.
Jeff links to Abrash's talks, which are well worth the time to go through if you have an interest.
I also have been keeping a doc that's a good resource and includes a fair amount of more technical references: https://randomfoo.hackpad.com/Virtual-Reality-7LTycEgSyp9
I think to some degree, the most interesting thing about DK1 isn't how "rough" it is, but more as a data point showing just how far things will have come in such a short period of time.
I bought Howard Rheingold's "Virtual Reality" 22 years ago! :-P
Also, even a bit of positional tracking helps a whole lot. MineCrift (Minecraft in the Rift) becomes a lot easier to play once you've got a Razer Hydra enabled with positional tracking.
The trick is to use it frequently for short periods when you're first getting used to it.
For some people, it takes strong guts to be willing to put through that practice that it requires. 10 minutes of Oculus Rift (dev prototype) will make me ill for hours. In addition to feeling like I'll to throw up, my eyes cant focus properly and I can't read text on a monitor afterwards. I'm not particularly motion sensitive.
I'm sure practice would help but I'm not willing to go through several days of nausea in order to be able to play 3d games.
I think that this is also the reason why Oculus isn't putting out their product quite yet. They need to get to a point where only a very small fraction get any kind of motion sickness. Otherwise they will be devastated by press reviews in mainstream consumer electronics reviews. Gamers might be less harsh about it. But if they get a "vomit helmet" reputation, that will be the end of it.
That being said, I'm very eager to get to try the Crystal Cove prototype with translation motion tracking (the IR LED + camera thing) and improved resolution. I think that better matching the head movements with the virtual reality will reduce motion sickness.
I very much hope that they will be able to roll out a better product but I will not touch the devkit again.
It's definitely a prototype, and you're right that it'll take new gameplay. People are saying that typical first-person shooters don't work at all.
The technology has clearly improved, not least that a decent phone now has more graphics power than an SGI Onyx of the era of the experiments, but nothing has magically changed about the people.
That said, could you be more specific about the problems caused with our current VR technology, and the reason they happen?
This comes somewhere on the tech tree after "practical flying cars" and "solving world hunger." But eventually, sure, why not.
> That said, could you be more specific about the problems caused with our current VR technology, and the reason they happen?
The most common and superficial health issue with VR (and AR) would be simulator sickness. The visual cues don't match what the brain expects via other sense data, so you get sick. It happens to some people much more easily than others.
More direct hazards:
Safety issues, e.g. tripping on stuff you can't see while operating a VR headset, or being distracted due to virtual stimuli.
Vision impairment, primarily in children younger than 6-10 years, but potentially also in older users with compromised vision systems.
Temporary visual impairment, typically after sessions longer than 20 minutes. Pronounced eyestrain due to unnatural focal behavior, object tracking, display quality, and optical alignment.
Nice papers to read:
Costello, "Health and Safety Issues associated with Virtual Reality - A Review of Current Literature" http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.6.3...
Viire, "Health and safety issues for VR". It gets cited around lots even though it's over 10 years old now, but I couldn't find a free copy. Look it up on ACM if you have access. You can preview the first page at the following URL. http://www.deepdyve.com/lp/association-for-computing-machine...
Stanney, "Realizing the full potential of virtual reality: human factors issues that could stand in the way." Starts on page 39 of the following PDF. http://www.researchgate.net/publication/232654060_Pen-based_...
You'll note that these are all late 90s. VR research was much better funded then. You can find more recent stuff though, at least with appropriate database access. More work did get done, but shoehorned under different keywords since "virtual reality" stopped being trendy.
> You are worried about supposed problems that came from 10+ year old tech. (from sibling's child)
Nope. The main difference between 10 year old VR headsets and the ones you can buy today is that the use of bulky CRT eyepieces vs. LED-based eyepieces. The middle ground being LEDs with gradually improving dot pitch.
The human issues remain more or less the same, seeing as nobody has re-engineered the human body in the last 10 years.
My grad advisor had been involved in VR & AR research going back - I forget, 20 or 30 years. So I did a fair amount of reading on the topic. You can make slightly better headsets today, particularly if money is no object - but the core limitations are really the human factors, and the timing constraints necessary to accommodate human senses and rate of motion.
Today we have experiments that bring limited sight to the blind and cochlear implants that bypass the ear are routine operations. I would say that gaming implants are a lot closer than you might think and are almost certainly closer than practical flying cars.
Solving world hunger isn't really on the tech tree, given there is already surplus food. It is firmly on the politics tree.
That's not true. All you'd need is batteries with higher capacity and some clever engineering of ducted fan propellers with variable geometry. You couldn't manage nearly as high speeds as passenger airliners in the lower atmosphere, but you could certainly manage cruise at speeds and efficiencies around those of a car. A modern sailplane cruises at an effective energy consumption of 3 kW (4 HP) at 100kph (calculated as sink rate at max L/D times gross weight). You wouldn't get this efficient without some really smart engineering of the lifting surfaces, but there are plenty of avenues that haven't been explored yet if you've got VTOL capability and don't have to engineer the wings for safe landing speeds.
The key is that the design has to use wings, it's the only practical way to generate lift efficiently.
Of course you can always get a microlight, but they already exist and they are not flying cars.
Current state-of-the-art VR tech is a huge advancement from late 90s VR tech, here's a few examples that run contrary to your curious assertion that the only significant VR tech advance of the last decade was the transition from CRT to LED displays:
-The usual inexorable orders-of-magnitude march toward greater processing power
-Commodification of smartphone hardware, which happens to be ideal for VR
-The ability to correct for distortion and chromatic aberration in software, rather than with bulky and complex optics
-Advancements in "sensor fusion", where multiple, complementary tracking sensor technologies are used in tandem, to compensate for deficiencies in any single tracking tech
-Low persistence technology that eliminates notoriously sim-sickness-inducing pixel smearing
You've clearly done your reading on this, but the VR landscape really has changed more in the last few months than the five years pre-Oculus.
Not true. While Abrash seems to makes this claim several times early in the PDF [1], near the end there's this:
> In addition to the question of how games will interact with input, rules about how players can move around a virtual space without getting motion sick or losing presence have yet to be figured out. We’ve found that traditional FPS movement is far from optimal and tends to cause motion sickness, so VR may be best with slow movement and a lot of up-close interaction, in which case we’ll have to learn how to create fun games around that.
Abrash admits that "traditional FPS movement" still makes people sick despite all the technical improvements touted earlier in the paper. His solution doesn't work for FPSs but instead proposes new types of games instead of the ones that people want to play. Flying fighter jets, the type of simulation that gives simulation sickness its name [2], does not involve the "slow movement and a lot of up-close interaction" that Abrash says is needed to prevent sickness.
Like you I was excited that Valve had solved the simulation sickness problem, but on closer reading found that it's just not true. Many people have the desire to use VR tech in virtual worlds doing things that involve normal-speed head movements without getting sick, but Valve has not solved this problem.
[1] http://media.steampowered.com/apps/abrashblog/Abrash%20Dev%2...
[2] http://en.wikipedia.org/wiki/Motion_sickness#Simulation_sick...
From personal experience and in watching a number of people in my office try out the DK1, I can definitely confirm that there are a number of things that are quite different when strapping on the Rift - relative scale of objects becomes much more important, world detail (books in bookshelves etc) takes on a much more interesting quality, and movement speed is definitely something that seems to scale down - feeling comfortable moving at walking/realistic speeds vs getting sick at traditional video game character speeds.
Too bad they didn't listen to the status quo and go about re-enginering the human body instead.
Cars in the 90s were less safe than cars today. You may build upon car safety studies in past decades but you need to test and re-confirm the conclusions of the older test subject hold true for the modern test subject.
Some things _may_ still hold true but where in science do you draw the same conclusion for a blanket category of "All VR"?
This will not work. The visual system is not an "information processing" system like a computer. In order to wire an "accurate input" into your brain, you would need to duplicate all of the nonlinear optical, electrical, and chemical transformations performed by the eye, in which case you might as well just use your eye.
In other words, we don't just see what comes out of the screen, so wiring that data into your brain will be gibberish at worst and a somewhat compromised new sense organ at best.
The resolution and the focus will certainly be solved long before an implant could give remotely similar performance, but an implant would sure be neat for the other reasons.
The case that used to scare the hell out of the lab techs I ran into was that of someone using a simulator at work for a few hours a day, but driving to and from the office. The context switching is far from instant, and acclimatising to it could be the eye based equivalent of giving yourself a calluses through repeat gentle stabbing with a needle.
Additionally, if you spend a good proportion of your waking hours in a device setup for you to focus at infinity, what happens when you take it off and need to focus elsewhere? The muscles won't have been exercised and you won't be used to using them when looking at objects you judge to be at different distances.
This is on top of the classic intro, which would be "welcome to VR - we're going to show you 20 new ways to make someone fall over with surprisingly little effort", and many of those don't even need head mounted displays.
As far as I could tell AR got a lot of early support largely because using see through displays eliminates most of the problems while preserving the most interesting non-gaming applications (i.e. keyhole surgery overlays on the patient) though the calibration requirements are insane.
Not to say you can't harm yourself with conventional setups, you can (epilepsy being the extreme example, but eye damage from using displays irresponsibly is very common in the geek world), just VR adds whole new dimensions to it that should really be far more openly discussed.
There is fighting a dragon with your D&D friends on a couch eating chips and what not, then there is fighting a dragon with your hands and body.
I'm not saying there isnt a place or this won't be popular, but I'd rather be in the OASIS somewhere with an immersion rig :)
There's a lot of potential there.
That was also true in the 1950s, 3d films were a success for a while back then.
Like the blog post said, VR people are after Snow Crash/Ready Player One.
I'm sure LARP'rs will want AR
This is by no means the only thing CastAR will be able to do - it's just what they have concentrated on in their demos. Perhaps because unlike Oculus, they can create an experience where many viewers can share the same scene (or a completely different scene) on the same physical surface.
I am particularly interested about CastAR for (flight, etc) simulation. The problem with Oculus is that it completely blocks your view to the real world. When playing flight simulators and other games with lots of controls, you need to see where your fingers are. E.g. playing flight sims with the Oculus, most of the time your hands will be on throttle and stick and everything is fine but when it's time to engage the landing gear, you'll have to peer under the goggles to find the button for the landing gear.
Perhaps the CastAR technology is more suited to things that pop out of the surface (like a board game button) than things that go deep into the surface (like a window). This might make it less suitable for simulation purposes.
On the other hand, the playing with the Oculus will inevitably be a solitary gaming experience where the CastAR may be able to preserve the social element in gaming. Of course, solitary and social games are targeted at different audiences.
But the best thing about CastAR vs. Oculus is the competition of two different technologies, which will inevitably drive the competition forwards.
1) How much harder and stranger will it be to do VR graphics programming?
2) Can we predict when we will be able to replicate real world in a graphics application?
3) AI has a end goal of achieving singularity. What is the ultimate goal of a graphics programmer? And how much closer are we to the goal?
Replicating the real world is still very far off, but there is an estimate where the resolution offered by the screen matches the average density of receptors in the eye. I don't recall exact numbers, but I seem to remember Michael Abrash having said something about it. Probably somewhere in the range of 8k displays per eye.
What's cool is that mobile manufacturers have already committed to creating screens capable of much higher-density screens than current technology. The VR space was a bit nervous regarding display technology due to the diminishing returns of higher-resolution screens but recent announcement makes hi-resolution VR not only possible but likely regardless of the success of individual VR companies.
> It's a big commitment to strap a giant, heavy device on your face with 3+ cables to your PC. You don't just casually fire up a VR experience.
This is the exact same problem that just killed 3D HDTV -- the "everyone has to put their special pair of glasses on now for this to work" problem. It turned out that most people just weren't willing to go that extra step of putting special glasses on to watch TV. And 3D glasses don't even have to be strapped to your head! They're feather-light and super-simple compared to something like the Rift. But they were still too much to win over a broad general audience.
(It could be argued that this is less of a problem for VR than it was for 3D, because TV content is more frequently consumed socially than computer/game console content is. But even if that's true, it seems like it puts an unnecessary cap on VR's ambitions; why wouldn't VR want to expand into the niche TV fills today?)
This is a big chasm for VR hardware, even very good VR hardware -- the more ceremony that is required to get from "hm, I'd like to have a VR experience now" to actually having the VR experience, the less likely it is that it will ever make the leap from early adopters to the general public. "Casually" is a good way to describe the way people interact with most media -- and that's only becoming more true as things like smartphones and tablets become prevalent. So anything that pushes back and tries to make that casual experience more formal is swimming against the tide.
The comparison with 3D TVs isn't so valid. In contrast to gaming, TVs are both casual and multi-user. Making sure everyone has the right glasses to watch 20 minutes of the Olympics (while conversing or eating or whatever) is much different than sitting down by yourself to do some proper gaming.
I beg to differ. "3D" failed in cinemas and there's no amount of commitment already given to that ensuring maximum enjoyment; travel, ticket prices, $10 popcorn, etc. I see the lack of compelling content, or at least a good enough reason to make said content 3D.
The same is not true of VR. Entirely new experiences are possible. I agree that it's an anti social technology but I see the solitary gamer or single individual getting a lot of value from this.
Games are actually made out of 3D geometry that you can freely walk around in and look anywhere so the 3D effect is on a ridiculously different level.
Convergence: What happens when virtual realities take over? http://pando.com/2014/02/16/convergence-what-happens-when-vi...
I know the author went to the Oculus Rift headquarters last year as they are big fans of his book.
Why do people jump to this worst case scenario for new tech ?
I predict that the first users of such setups will be ones currently sporting 4+ monitors.( stock brokers come to mind ).
When OR becomes cheaper than the equivalent set of displays you will probably use it.