The Problem with Reality | Hacker News Reader

The Problem with Reality

storystudio.oculus.com

128 pointsrunesoerensen119 comments

The problem with all current VR (that I'm aware of at least), is that it models vision as if our eyes were permanently stuck forward and that all of our looking around was done by turning our head.

It's not remotely correct, and I personally think it's what makes people sick.

Our eyes are the viewpoint control, the head is a very coarse grained pivot, but it just controls the boundaries of the viewport. The view is always centered on whatever our eyes are looking at regardless of head position.

You can read a sentence while shaking your and twisting your head. Try that with an oculus (or any other headset).

In fact it's actually very hard to simuluate with your body the kind of sliding view that headsets give. You can do it, but it's hard. I personally have to totally defocus my eyes to stop them from locking on to something as I turn my head.

The minecraft hack that snapped view changes to 15 degree increments is actually much closer to how things work biologically.

This, and the fact that you cannot focus into the distance. In real life most of your view is blurred except the part you are focussing on. In VR it still just looks flat, because if you focus on something in the distance, nothing happens.

I wonder what happens if one uses VR goggles for too long. Will one lose the ability to focus?

If you wear prisms that invert the world, your brain will eventually see the world inverted when you remove the prisms, and this effect will auto-correct after some duration.

It also gives you one hell of a migraine.

Things might work differently for small children who wear VR goggles for extended durations. Their brains might get wired the wrong way, and there might be no "auto-correct" in this case.

I have a PHD in small children vision, I concur.

why would an adult who grew up with normal vision be able to adjust to something else but not the opposite?

Adjusting to the inability to focus is different from learning to focus. Like the jellyfish who can't orient themselves properly if they were born in zero gravity [1].

http://www.businessinsider.com/space-born-animals-adjust-to-...

I don't think it would be catastrophic, unless these children spend 90% of their time playing VR games. I would expect it to be similar to exposing a child to a second language. If you replace their exposure to the previous language completely with the new one, their proficiency at the first one will suffer but if you provide enough time for both, the brain will improve in both.

I'm watching developments with light field displays for this with interest.

That said, you don't really notice this effect, or much else, once you're immersed in something. I've hundreds of hours in the vive and I still punch walls, the ceiling, lean on objects that don't exist.

The point is that regardless of the various things that would make it better (4k light field displays, bigger fov, higher refresh, blacker blacks, etc etc) the tech is good enough today - the fact that one is able to surrogate away from "I am stood in my living room wearing some silly goggles" to "I am stood in the snow atop a peak watching the sun set, I should get inside or I'll get cold" is the litmus test. Literally everyone I've jammed in it, young and old, has lost themselves in it.

I'm really curious if this is based on your actual experience with a Vive or if this is an assumption based on what you know about the technology.

I read an article at some point that claimed this was an issue so I consciously tested it in the unit - focusing on something near my face and then farther away without moving my head and it actually feels (to me) pretty correct.

I don't really know the science behind this versus light field displays or whatever, but if you have access to a unit I encourage you to try it - it's very difficult to detect issues with focus and it absolutely does not look flat (again, to me).

I would be very interested of all the opinions in this thread which of them are from people who own a unit and have tried these things and which are assumptions.

This technology is so incredibly convincing I'm embarrassed to say that I fell over trying to lean on a virtual pool table at one point. I simply forgot that it wasn't a real thing. :)

There's no focus in any VR headset currently. So whatever you experienced was not that. You might be thinking of the stereo convergence, where the left and right images come together where you look at them. But they don't have any capability to adjust to the focus of your eyes right now.

> I personally think it's what makes people sick.

It is possibly 'one' of the factors that make people sick but that doesn't preclude the other very common issues that make people sick like artificial locomotion. After giving ~100 VR demos I haven't found anybody that gets sick with non-artificial locomotion games (including my mum who can't even watch a video game on a TV without getting sick).

Yep, this is the problem. And we've known about it for more than a decade.

I tested using VR when we were building the Virtual Production system for Avatar back in the summer 2005. We were seeing if it would help the actor's eye lines to actually see the virtual world they were performing in. The problems then are still the problems now. Nothing meaningful has changed with VR yet.

- Lack of convergence. (Movies, done well, are converged on the subject, VR is not)

- Lack of depth of field.

Latency, fov, and resolution were also issues but to a much lesser degree.

No one will be able to use VR for long periods of time until those two issues are resolved. In fact, I think it is actually irresponsible to let anyone under the age of 12 or so use VR without correct convergence and focus due to how it may effect ocular muscle development.

Background: 100+ hours in Vive, hobbyist VR dev, and 20 demos or so

I see two kinds of VR "sickness".

1. There is sickness caused by locomotion implementations. Testing seems to indicate there are some people who will just never adjust to these. I really don't think FOV and eye tracking will fix it. I've experienced locomotion sickness in some games, and in all of them I've been able to completely adjust in 1 to 2 hours. That adjustment seems to be relatively permanent as well. I don't think it is a rare trait, but it seems that not everyone can do that.

2. There is this sort of initial "VR Fatigue" where the eye tracking issues might fit in. I've noticed a trend the first or second time people use VR for a real session (1 hour +), it is somewhat common (50%?) they get a little nauseous. I've really just been chalking that up to "new stuff is weird" and the heat the headset puts out. This sort of sickness seems to universally fade quite quickly for people, and could have to do with the head on a pivot thing. From what I've seen, most people find the sweet spot/head pivoting natural almost immediately.

TL;DR I spend a lot of time in VR and with people in VR. Eye tracking or larger sweet spots is near the bottom of the list of problems/discomforts.

Nausea inducing incongruous locomotion is of course an absolute fail. But that is not a problem unique to VR, and for the most part you know it going in if you're likely to get motion sickness.

The actual issues with VR of eye fatigue are more subtle, and often hard for users to identify. They simply get an increasingly negative attitude towards the experience over time, and over repeated use. This is why an entire industry can spin up and have what seems like a great product, but have an odd lack of audience.

The exact same thing happened with stereo in movies. The technology existed for a long time, but lost favor because it was a subtly unpleasant experience. Avatar and the modern resurgence of stereo has been more successful because of the decision to always converge the subject. Even now many supposedly expert "stereographers" try to emphasis the "stereo" effect by pushing the subject into a different depth from the screen by not converging on the subject.

Watch a stereo version of Avatar without 3d glasses. You'll find that 95% of the film is completely watchable without glasses because the subject is always converged, or the shot isn't even stereo.

I get motion sick in some games, notably the Half-Life series, unless mouse input smoothing is enabled. I also get motion sick in cars sometimes. Would this be the former type of motion sickness?

I have a little trouble understanding what you're trying to say here but I don't think it's really accurate. Positionally tracked headsets like the Vive and Rift do not model vision as if our eyes are permanently stuck forward. They are very much intended to be used with normal eye movement within the field of view of the headset while the head is stationary.

Both are low persistence displays with a 90Hz refresh rate. Low persistence was an innovation from Valve explicitly designed to reduce blur and ghosting resulting from eye motion relative to the display as you get when fixated on a stationary virtual object while turning or moving your head or when trying to track a moving virtual object with your eyes while keeping your head still.

In an ideal world we would have even lower latency head tracking and higher refresh rates but current headsets already allow for legible text when moving your head thanks to low persistence.

I'm not familiar with the Minecraft hack you mention but approaches to snapping view changes are generally designed to address issues with using a controller rather than actual 1:1 head motion to change view orientation. View rotation independent of head movement tends to produce nausea due to the disconnect between visual cues and cues from the inner ear but really has nothing to do with any supposed assumption of eyes being stuck forward relative to head orientation.

Issues with convergence, accommodation and depth of field raised in other replies do still exist with current display technologies but they are again not particularly tied to any assumptions of fixed eye orientation.

It's more like you're wearing goggles than "fixed eyes", you can still look around, your field of view is just restricted because of hardware limitation at the moment.

This also is not a common reason for motion sickness in VR, that's just incorrect.

FOVE is the first VR start up working on eye tracking for this very reason. http://www.getfove.com

> The minecraft hack that snapped view changes to 15 degree increments

Link? Haven't heard of this, and I sometimes have nausea when playing.

> You can read a sentence while shaking your and twisting your head.

I tried to read the rest of your post while doing that, and it gave me motion sickness. I'm not usually susceptible to it so I guess that says something. Maybe reality is actually as bad as VR?

There's also less to render, if you only render where you're looking. You can render lod proportional to retina receptor density.

Can be done with eye-tracking, or contact lenses.

Big the big problem is latency: it has to be even quicker than what current VR headsets can get away with, especially for saccades.

We may have direct visual neural implants before we have that.

> You can read a sentence while shaking your and twisting your head. Try that with an oculus (or any other headset).

Actually, with the issues with small fonts (and subpixel gaps in the DK2) in Elite: Dangerous - one suggested fix is shaking your head from side to side for easier reading of text, and it works fine?

Try wearing progressive bifocals. I think you will find that the experience is very similar. You have this little window where you can see stuff, but it has very small angle of visibility. In the top half you can focus out to infinity. In the bottom half you can focus up close. If you look in the wrong place, there is absolutely no way to bring what you are looking at into focus. For the vast majority of things, that means you have a single focal length (infinity) and you bring the thing into the visible part of your gaze by moving your head.

The thing that is really strange is that within a few weeks of wearing progressive bifocals, everything just looks normal. Your brain compensates. Or to put it in a different way, your brain is always compensating so that while you think you have great vision, the reality is that your brain is paving over all of the badness. It really does an amazing job even with something as unbelievably bad as progressive bifocals.

VR does not need to provide the same experience as reality (without glasses). It just needs to provide something good enough that the brain can be trained to pave over all the badness. I've only ever played with VR headsets at a store, but I can tell you that they are already better than my bifocals ;-)

That's why I look forward to StarVR [1] from Starbreeze Studios (or what the competition will do if it hit's the market. I'm a bit skeptical if Starbreeze has enough resources), since it has a field of view of 210 degrees, which allows the user to look to the corner of the eye and see stuff like in real life.

It seems in the next prototype they want to also integrate eye tracking, which hopefully works great together. The downside of the current prototype seems to be the weight, only 60 hz refresh and hard to make optics which have a tight sweet spot, and if you are not in the sweet spot it looks bad.

[1] http://www.starvr.com/

Imo the biggest problem with VR is lack of controls. All of the games I've seen (that aren't driving/flying simulators) use same "teleportation" movement gimmick with some slight leaning/ducking or they just have you stand in place while stuff happens around you.

Not to be too much of a downer, but I still think the main problem with VR isn't the technicality, but the practicality.

We had the exact same problems with motion controls. They might momentarily enjoy the novelty of a motion game, but it's just a gimmick. When you get down to it, a lot of people enjoy just sitting on the couch twiddling their thumbs to get the character to do all the work.

They play games to relax, not get a workout. If you force them to stand up and move, you've already failed.

Where I do see VR having some interesting applications is in the workplace. Replacing all your monitors and work environment with a virtual space of unlimited scope.

Imagine remote working - where one moment you could be typing away on a sunny beach, the next you're in a scheduled meeting room with all your colleagues, without having to move an inch.

Agreed. I would invite VR developers to consider whether or not home is the best setting for VR gaming. The developers of Killer Queen [1] were smart in developing their game for arcades; had it merely been a Steam release, it may have been looked at as just another quirky indie game with pixel art.

Come to think of it, VR in arcades could actually take off. Think of a physical enclosure designed to contribute to the presence of the VR experience. Arcade rail shooters have been accomplishing this for years through the use of a chassis that moves based on events within the game.

[1] http://killerqueenarcade.com/

I've come to the same conclusion: VR belongs in an arcade. The tough part is coming up with a game worthy of a VR arcade... Batman Beyond-style.

If you want to do arcade VR you'll need to solve the Pink Eye problem before I get anywhere near it.

There's a pink eye problem with HMDs? This is the first I've heard of it, but I suppose it makes sense.

Yep. A local company in OC is currently developing a VR headset condom that will solve the hygiene problem. Going to a bunch of meetups and trying out VR can get disgusting.

I'm all up for anything that ushers in a revival of coin-op arcades outside of Japan.

I like the idea of taking old laser tag arenas, kitting them out with lighthouses, and then having vives hooked up to backpack pcs. Battery life is only an hour or so, but I'd like to see you last an hour of running around a jungle/glacier/moon/starship/wartorn village. You map the physical space into a variety of play arenas built in your engine of choice, so that snowdrift corresponds with that pile of foam, this bridge is that ice bridge, that room is this room, etc.

The tech is already here - the hurdles are pretty minimal, it's just a question of setup.

https://en.wikipedia.org/wiki/Dream_Park

The Oculus and its ilk happen to work great for certain classes of games - just about any vehicle-based simulator (racing, flying, etc.) where your immediate reference frame is static is perfectly suited to a 3D headset. I hesitate to call it VR in that case, as I feel like freedom of movement in the real world is wrapped up in some definitions of VR. For the things I care about, I essentially see 3D headsets as an evolution of the TrackIR concept (https://naturalpoint.com/trackir/)

Yeah I just saw this snippet on reddit:

https://imgur.com/topic/Gaming/swpjI58

That does actually look like a great use for VR in gaming. Stationary character, mobile viewpoint.

Sorry, I don't agree. It's a gimmick only if you don't appreciate the immersion.

I can't imagine playing Xortex (a game in Valve's 'The Lab') on a 2D monitor. It just wouldn't be possible to even come close to replicating the experience.

Valve also has a new product called "Destinations" which is basically creation tools and a viewer for photogrammetry scenes. One of their examples is Mars, with a realistically rendered model of Curiosity on the surface (which itself is based on photos from Curiosity). It's the closest I'll ever get to walking on Mars. And being able to walk around an actual-sized Curiosity is quite something in itself.

Yep. TV's are not great at conveying radio plays. But, that's all people could think of to do with them initially. Turns out they are better for content like Game of Thrones. Phones and tablets are not great at playing deeply engaging games like Quake III, but people tried to make that work initially. Turns out they are better for shallow games to fill in spare moments of boredom where ever you are. VR is not good for a lot of games we are used to. This is the experimentation period where we get to learn what it can do differently and discover new styles of games that didn't make sense to attempt before.

> Yep. TV's are not great at conveying radio plays. But, that's all people could think of to do with them initially. Turns out they are better for content like Game of Thrones.

I get what you're saying, but I think that's a rather terrible way to contrast radioplays and content made for TV. The experience of watching "Game of Thrones" isn't that different from listening to the BBC radioplay of "Lord of the Rings"[r]. I think "The Wire" or "Generation Kill" might be better examples of things that I think work better on TV, than I can imagine them doing on the radio. Or maybe "The Expanse" or even "Firefly".

Doesn't mean I don't like "Game of Thrones", I love the adaptation - but with its many characters and demanding storyline (even for those of us that have read the books) - I don't know. Somehow, while it is a beautiful production, I think it could've worked with a similar script for radio.

[l] Apparently the original series, in its 13 episode format isn't available to download/buy - if I understand it correctly the following Amazon link is to the remaster: https://www.amazon.com/Lord-Rings-Fellowship-Ring-Dramatised... See also: https://en.wikipedia.org/wiki/The_Lord_of_the_Rings_%281981_...

A search did turn up this: https://soundcloud.com/inkmore/sets/lord-of-the-rings-radio

Which I assume should probably be taken down due to copyright infringement.

I am really only willing to continue a conversation with someone that says VR is a gimmick if they have actually played through a number of room-scale titles. Mostly Budget-Cuts, The Gallery, Minecraft in VR, Audioshield, or something like that.

There is still a large sentiment that compares this technology to the Wii style motion controls, and they really aren't even close... at all.

There's the problem though. I don't think all that many people are ever going to get to play a room-scale title.

First there's the expense of the headset. Granted, it might come down a bit, but it's still a large expense on top of a console.

Second, there's the space that you need to play a Motion + VR game. It's not trivial for people to get that. A simple console + TV setup can fit in any tiny apartment around the world. Dedicating a whole room just to play a game? That's like asking people to move house just to play VR...

Even in my house which is fairly spacious, I'd still need to move around furniture everytime I want to play. It's just not going to happen.

Motionless VR, or at least, seated + arm movement VR, has a fair amount of potential however.

Eventually mobile phones will gain the power to provide compelling experiences, and tracking technology like Lighthouse (or low-latency SLAM tracking done by the phone) will allow for room-scale.

The headset itself will basically be a plastic holder with lenses.

Google is already preparing for this future with Daydream (https://vr.google.com/daydream/).

Yeah, that solves the price issue maybe, but it doesn't solve the space issue for Motion VR. Even in quite expensive houses, you're still going to be moving furniture to play a game. It just doesn't have the same ease of use of sitting back on a couch playing with a controller. I just can't see Motion VR growing beyond a niche market anytime soon.

The best idea I've heard so far is VR arcades, that would be fun.

I do it in a fairly small apartment without moving any furniture.

Just because you're not able to do it, or unwilling to adjust a few things, doesn't mean others won't. HTC & Valve have sold nearly 100,000 Vives, Oculus probably similar.

> The best idea I've heard so far is VR arcades, that would be fun.

I'm waiting for laser tag arenas that are fully mapped in VR. Walking up real stairs while seeing them in VR... What do you call it when it's past room-scale? Building-scale?

Even with the initial bugs with Elite:Dangerous and the DK2, I have a hard time engaging with the game in 2d - even now that I have a 30" monitor. Being able to look "up" through the roof windows of the cockpit and admire the huge yellow sun (or whatever) hanging there, makes a world (universe?) of difference. As for the "gameplay" that they focus on (dogfights and the like), I couldn't care less. I'm happy to be a glorified trucker, if I get to do it in space :-)

How much will it cost to be able to view this Mars experience through the VR and how many times can you do it before it get's boring?

> Imagine remote working - where one moment you could be typing away on a sunny beach, the next you're in a scheduled meeting room with all your colleagues, without having to move an inch.

I'm actually eagerly looking forward to what I guess would be called Fantasy Tourism.

VR apps that let you explore or even live in fully fleshed-out fictional locales. Of course it would probably just start off as Virtual Tourism, where you can remotely visit real-world cities and landmarks and interact with other visitors there, before adding beautiful, bizarre or surreal places that don't exist (or can't exist) out there.

I grew up and live in a fairly squalid corner of the world, and seeing pictures like [1] [2] [3] [4] fills me with a wistful longing. I'd totally spend most of my waking hours in these if I could.

[1]: http://i.imgur.com/F13RGuh.jpg (courtesy of 防人 on pixiv)

[2]: http://i.imgur.com/AKV5QkR.jpg (Raphael Lacoste from DeviantArt)

[3]: http://i.imgur.com/jlbAXdq.png (三登 on pixiv)

[4]: http://i.imgur.com/I6psL2j.jpg (by Ford Nostalgia)

No doubt popular brands will jump in sooner or later and start offering tickets to say Tatooine or the Shire.

They don't need to have any "gameplay" per se, but they should still have realistic physics (more than most proper games), be "staffed" by real people (running the in-world shops etc.) and of course offer lots of character/cosmetic customization features.

Whoever makes the first killer application of that idea will probably become THE social hangout for the kids of tomorrow to spend their time in, maybe killing Facebook and the like. That is, if Facebook doesn't come up with something like that themselves.

Now that's a cool idea! Not sure I'd want real people in it though, might end up a bit Second Life-ish. But if you could each get sharded into your own version of the world, that would be a cool experience.

You should try InWorldz. It doesn't use a HMD yet (though there has been some work done on a true VR client), and the graphics quality and physics may or may not be as perfect as you want, but it fits what you are looking for exactly in the social aspects. There is no 'game' per se, other than whatever contests, roleplay etc. that users might choose. Some people do spend most of their waking hours socializing there, or running real businesses there, etc. All the content is created by users.

https://www.youtube.com/watch?v=w4KvpOrSNLU

You also lose the social interaction that gave motion controls their (limited) success. Wii Sports is fun because it's a light social activity like card games or horseshoes. VR is currently an anti-social activity, even moreso that normal video games.

VR is as anti-social as a telephone.

If you are hanging out with friends, pull a little box out of your pocket, press it against the side of your face and start talking to some invisible people and acting as if the people right in front of you don't exist... then you are being a jerk.

But, what if you were physically separated from your friends and wanted to hang out with them. Imagine if you could pull out a little box, press it against your face, start talking to them and suddenly it was as if they were right there with you no matter where they were on Earth. That would be a tremendous social enabler.

VR is like the telephone, but more so.

That's my rant anyway. On a lighter side: You are right that hanging around someone who is playing a game in VR is no fun. And, currently most VR games are still single-player because there currently isn't enough market to support the budgets required for many good networked multiplayer experiences. But, social VR is so awesome that there is no doubt that it will be a major use, if not the majority of use of consumer VR.

Not sure if you've experienced watching somebody play in VR but I personally find it really fun. This video gives you a good idea of it: https://www.youtube.com/watch?v=qYfNzhLXYGc

The networked social interaction is more social than any previous networking technology. And in person, the Vive usually does about as good at wii sports for keeping everyone engaged, even those who aren't playing.

This is just plain wrong. Please see Rec Room.

Even if you're not playing a multiplayer VR game, the people around you are having a blast watching and playing along with you. See the thousands of YouTube videos where people play VR with friends recording.

  > This is just plain wrong. Please see Rec Room.

no it's not wrong. I bought the GearVR in anticipation of getting the Vive when it launched. Wearing goggles is far more anti-social than regular gaming. It totally cuts you off from family.

I have a 2 year old and wife at home. During regular gaming, if my son or wife want my attention I am aware of it and it's trivial to pause and give them the attention they deserve. Goggles on the other hand are fundamentally isolating. It's hard to describe, but others that aren't sharing the experience with you feel cut off and almost hurt. Maybe that's not the best way to describe it but it's close.

The experience has put me off getting VR at this stage.

> Replacing all your monitors and work environment with a virtual space of unlimited scope.

I don't even like wearing ear buds or headphones all day. There's no way I would last all day wearing a visor. I think for specific tasks it's pretty useful, but not for all day everyday.

This is sort of true, but also not.

Besides, I really enjoy playing 2D games in VR through Steam in that giant theatre room.

They might momentarily enjoy the novelty of a motion game, but it's just a gimmick. When you get down to it, a lot of people enjoy just sitting on the couch twiddling their thumbs to get the character to do all the work.

It's not at all like a Wii style motion control. Gestures aren't merely mapped to button presses. You have precise control, so you can do something like swing a sword between a monster's shield and helmet.

That said, if the game is reducing the motion controllers to something like the Wii motion controls, then it is a gimmick, and you might as well be sitting on your couch.

I also see interesting applications as a passive device. It allows you to attend a concert or a sports match, and view things from the best possible spot in the audience. This alone, I think, could be a multi-billion dollar market, if only the device (and the content) was cheap enough for the average person.

Yeah that's the way I see it going too. Anything where you can just kinda sit there and not worry about moving much other than your head and arms is probably a great match. So virtual work apps, VR tourism, VR concerts and sport, racing games, etc.

It's just the whole movement thing I don't buy.

They've really got to think about the size of their market as well. Time and time again, any gadget or gizmo with limited hardware penetration has failed in the past. Motion + VR not only requires some fairly expensive hardware, but also requires a large clear space in your house somewhere. They're heavily limiting their market with that. No big publisher will seriously risk making a AAA game with such a small market.

Motionless VR has some serious potential though.

I'm curious. Have you actually tried the HTC Vive? Because my experience with it is entirely the inverse: I get bored playing flat games now. I just want to go back into that VR world.

This, if you haven't tried the HTC Vive (or the Oculus with the Touch controllers, but mostly the Vive), then you haven't tried VR.

There is no doubt in my mind this is no gimmick, it's the next step.

Exactly this. The only "flat" games I can now bear to play are the likes of civ. Playing something like fallout 4 on the TV now feels like I'm looking at the world through a squashed little tableau. It's dull.

It seems most who are calling it a gimmick either haven't tried it, or tried the dk2, or tried it at a trade show and haven't tried living with it.

I use it daily, for everything from a workout in holopoint to racing in project cars (all non vr driving games are now ruined for me, I of course picked up a wheel etc.), from minecraft to playing table tennis. I never thought I'd one day be in the minecraft world I started many years ago.

Playing greenwater at the moment and having to put it down frequently, as code has never made me jump and scream before.

Anyway. It's not a wiimote type gimmick. It's the future of human interfaces.

What do you mean when you say we had problems with motion control though? Because Nintendo certainly made a killing on it. It's not so popular now, but it was hardly a failure

Oh my god I love this idea. We have 3 monitors where I work and it STILL doesn't feel like enough. Having an infinite area to place a Visual Studio window, SSMS, etc etc would be amazing!

Keep an eye on Envelop: http://uploadvr.com/unlocking-mystery-envelop-vr.

I'm not involved with them, at all. Just a fellow enthusiast waiting for this technology!

In my limited experience with it, and not being a gamer, I think it has great potential for mindfulness and quasi-psychedelic experiences. I tried a simulation of scuba diving and after a minute felt my whole body and mind relax into a peaceful state.

It would be nice to have something like that by my desk when I'm getting stressed out from work. Take 15 minutes and visit the tropics, Antarctica or the Andromeda galaxy.

When used appropriately, I really liked motion controls.

Lag needed to be improved a bit, but I think Metroid Prime and Resident Evil 4 on Wii had the best FPS type controls ever. Even with lag it was really fluid. Headshots in RE4 Wii were so easy.

I hope modelling/engineering/architectural/etc software gets a hell of a lot of VR integration, that's always seemed to me the perfect application - it allows for an environment that close as possible the real thing without losing the technical tools that allow for rapid prototyping. Thinking of stuff like Autodesk Inventor, where in 2D it feels almost like you're milling physical items, with the ability to morph or clone or whatever at will. With VR, the development experience could be phenomenal.

I'm really not a fan of the external camera. IMHO the ultimate solution is for a camera (or two) on the headset running SLAM to create a virtual world that overlays the real world. Then the experience has to be designed to map onto a large variety of real world environments. You're still free to change the textures and lighting of course. While this is not a generally applicable concept, it would make the virtual experience vary depending on your real-world location. Imagine going to a friends house to play Doom because his house is actually more fun.

You're referring to Mixed Reality, which is the concept that Magic Leap is working on (rather fantastically). It may ultimately prove more enjoyable than VR in a variety of ways, but it has a different set of challenges. The goal there is to bring virtual objects to your world, while the goal of VR is to transport you to another one.

Augmented Reality or AR is a better word.

Imagine going to a friends house to play Doom because his house is actually more fun.

That reminds me of the days when kids could make maps of their friends' houses and schools in various 3D games without being labeled as a psycho killer.

http://www.theregister.co.uk/2007/05/03/student_counterstrik...

Or playing with nerf guns in a friend's house?

We used real air guns, but loaded with paper balls once while celebrating a school year's end, was lots of good laughter :)

I think we're going in that direction, but current SLAM techniques aren't fast enough to provide the extremely low latency needed. John Carmack has mentioned prototypes that mostly work as long as you have dedicated room markings, but it gets unreliable when the environment can't be controlled, so it's a long way from consumer-ready.

If it could be done now, JC would definitely be getting it into the next GearVR. The other problem is that even with current best-case everything, something like GearVR doesn't have enough processing power and if it did, it would kill the battery.

The Vive already has a front-facing camera that it uses for Chaperone and a few other things, but it's not involved in tracking at all. In the short-term, expect other HMDs to borrow this idea.

Have you seen the HoloLens by Microsoft? They've developed a custom silicon chip to off-board all the SLAM processing, and they use that for realtime localisation. Current SLAM techniques are absolutely fast enough for this, VR manufacturers just haven't gotten on board yet.

That's just how you get started. Then what? How do you move? You usually can't move very far in the real world. Although it would be fun to have VR events on big open spaces, so you could.

The U.S. Army has tried VR big spaces.[1] It works. Their people wear knee pads and helmets, so falls aren't too serious. The Army also accepts that getting banged up a little in training is a normal part of life.

This isn't going to translate to the living room.

[1] https://www.youtube.com/watch?v=V34gCw4fyLs

There are plenty of locomotion solutions in VR. Simple teleportation works the best. The best implementation of teleportation that I've seen is in the game Budget Cuts:

https://www.youtube.com/watch?v=TwtEw2ggPnA

Wow, genius!

Though I wonder, why most of the VR games I see now keep showing tools you handle "floating in the air"? It's a pretty immersion-breaking element, I think.

You mean how there are no hands and arms attached to the guns? While it looks funny on a video like this, it doesn't bother you a bit while you're in there.

My best explanation is your brain doesn't really care about your limbs while you're holding something. Your attention is on the object.

Rendering fake arms and hands would be a bit off-putting, I think. VR isn't tracking the position of your elbows or fingers, so it would have to guess their positions, and pretty much anything that's doesn't match reality gets noticed by your brain.

Rendering nothing instead of the wrong thing is more immersive.

It bothers me. I was playing with the Vive and the controllers with the smooth trackpad and it was a really odd experience.

First I couldn't tell what I was doing as the pad was frictionless. Then I saw a little red light to indicate I was touching the pad and it would move around in a circle. Okay that was helpful.

Except when looking down and it I was weirded out by knowing by hands are there in real life but I couldn't see them. What the heck was going on? Shouldn't I be seeing myself there but I'm not?

It was sort of like looking into a mirror and not seeing a reflection and wondering if you're a vampire or something. Yes that sounds crazy but it was quite odd.

That works very well, until you're inside a virtual room that's bigger than your real room. Then it's a pain.

Did you even watch the video? That's someone in a tiny apartment room (about 2 meters squared) playing a game that spans an entire building, with pretty much every room in that building being larger than his.

I can see that this is a good solution, but I kind of still dream of a VR world I can run around in... I hope humanity's future isn't just going to consist of standing in a single square foot, using teleportation to move around :(

I was just musing with my girlfriend about the logical progression of immersion technology, especially as it applies to games.

Right now, we're still trying to perfect the visual part of VR. In 5 to 10 years we may get to the point where graphics are no longer indistinguishable from reality as we know it.

Research into providing the sensation of touch has begun already and in, say 25 years as a safe estimate, we may be able to perfect that as well. Games where you can comprehend the texture and weight of every object, wince from every blow you receive, and even feel a cool wind or gusts of heat all over your skin.

In about 50 years we will have the senses of smell and taste covered as well, and by 100 years from now VR may become impossible to tell apart from reality.

But.. people will still KNOW that it's fake, because they will remember being out here in reality.

So what's the next step for achieving maximum immersion?

Voluntarily wiping out your memory of the real world before you log into that next-gen VRMMORPG. Eventually you have people being born into a world without a clue and having to be cared for by and learn from others until they can orient themselves, and all sorts of theories about what happens when they "die."

> In 5 to 10 years we may get to the point where graphics are no longer indistinguishable from reality as we know it

I feel like I've been hearing this statement for decades now. The human brain is remarkably good at pattern separation, and given enough time and exposure to a stimulus it will work out what is real and what isn't. Until we have a simulated reality that is as information rich as reality we'll always be able to tell the two apart

Multiply the placeholder times with the estimates of your choice, of course :)

This. Does your game have a river? If so, humans observing the river will veeery easily be able to spot it's a fake. Because an actual realistic simulation of turbulent river flow is way beyond the largest supercomputers today, and Moore's law has run out of the steam that has made todays consoles have the computing power of your fathers supercomputer.

I'll bet dollars to doughnuts that even twenty years from now, by far the cheapest way of creating a completely photorealistic VR river scene is to build the actual river and record video of it. Then you "only" run into the problem of storing and distributing the petabytes of data you record.

Yeah, Witcher 3 is one of the best games I've ever played and has some of the best graphics too. The characters are simply amazing and the general look and feel is superb.

But... look at the ground and you'll just see a flat greenish noise texture. They try to cover it with clumps of grass and vegetation, which works to a degree, but not completely.

Look at the water flowing - particularly on any kind of whitewater rapids/waterfall - compared to the character textures and the buildings, it's really quite disappointing.

You need a fairly complete simulation to really fool the human eye and we're a long way from that.

Game (world) design (and art in general) is about accommodating and working with the constraints of the medium. Make a realistic (or fantastic) desert game today, if you can't get the water right. Make a great simulator for the oculus (eg: Elite: Dangerous) rather than trying to make a roller-coaster simulator if your user won't be able to get multi-million-dollar 360-degree rotating simulation chairs (think NASA style stuff).

As for a "VR" river, there's nothing wrong with filming a real river with a 360 degree camera, and then augment that scene, rather than "building a river". A real river on a virtual Mars, might look better than a virtual river on a virtual world.

Of course you can dream up a game that hard to bring to life exactly as you envision it using current technology. That doesn't mean new technology doesn't open up a few new possibilities. If you run into problems thinking big and complicated, try thinking smaller and simpler. I'm sure a Populus style game would be a lot of fun with the DK2 (isometric sandbox god simulator game with slightly cartoony graphics). Why does it have to be realistic?

Technically true, but I don't think it matters. Sure, the brain can easily tell reality from our electronic media apart, but very frequently it choses not to. We don't need super photorealistic graphics for that - I've personally been sucked into games and movies with anime characters, and my brain didn't care. As long as the virtual reality is somewhat consistent and doesn't mix with the "real reality"[0], the brain can accept it as real - especially if you distract the player with a good story.

[0] - keeping out the "real reality" from showing up involves obvious points like keeping distractions (especially other people) away, but also some meta-points; if your product has glitches or even simply keeps "breaking the fourth wall", it may be just enough to prevent any kind of immersion from happening

I actually think the games/virtual environments that avoid attempting to recreate reality will be more immersive initially. Being in a cartoon world would probably avoid distractions about something not looking "realistic" because as long as it seems consistent to the world you're in, I'm guessing it'd be very immersive. If I'm in a Toy Story world (or watching the movie), I'm not really bothered that the physics of how a character moves doesn't seem to match any reasonable muscle structure because I have no expectation that it should.

One one hand I agree with you. On the other hand, looking at how far movie/tv FX has come, like http://nerdist.com/amazing-game-of-thrones-battle-of-the-bas... - I'm not so sure.

10 years ago, the state of the art was something like Nvidia 8800 gtx - it looks like the recently launched 1080 is a handwavy 4-10x in terms of performance[b]. ~10x the RAM, but not bandwidth etc. Having recently(ish) upgraded from early 2000s system to an i7 with an AMD R9 -- I would anecdotally claim that we've come quite a long ways in ~10 years. Even if part of that is "just" making 2k-4k gaming a reality.

I think it is a little bit like optimization of performance. Small changes in aliasing, changes in resolution, refresh rate -- they all add up to move us closer to "the real". And I think one shouldn't ignore the work done on the modelling, skeleton, animation and physics side either. It all helps make artificial worlds feel "more real".

With all that, I don't really see why "realistic" is such an interesting goal. For simulation (be that virtual tourism, or a driving simulator helping people become better real world drivers) - sure. The tourist want beautiful scenery, the driver wants real physics (and neither would mind getting both).

But for gaming -- chess isn't very realistic, go is rather abstract and tabletop roleplaying games have the best graphics engine - shared with literature - the human imagination.

https://www.game-debate.com/gpu/index.php?gid=1701&gid2=879&...

> So what's the next step for achieving maximum immersion?

Do we need to go further? The classical suspension of disbelief already works pretty well with television and video games. The most important requirement is to keep distractions away - especially other people. That's personally why I personally tend to watch movies and play storyline-heavy video games only late at night.

The brain is very good at recognizing and telling apart patterns, but it's also very good at accepting them - including whole invented realities, as long as they're consistent and don't constantly mix with the "real" one.

"So what's the next step for achieving maximum immersion?"

Just walking outside.

I'm glad that racing sims and flight sims don't have to solve these problems, because VR flight sims are exhilarating. Only thing that's needed is G-force.

The podium/teleport concept seems very intuitive, at least it definitely was in my mind before the article explained it. Lots of interesting gameplay mechanics still need to be solved for VR, interesting times ahead.

Personally I think the best solution will involve some sort of hardware that can keep the player stationary while giving them the feeling of walking around freely. I'm not sure exactly what this will look like, it's generally not a good idea to walk on your treadmill while blindfolded.

I agree. There's actually already something in development called Virtuix Omni[0] that does what you think.

[0]: https://en.wikipedia.org/wiki/Virtuix_Omni

Prior art: https://en.wikipedia.org/wiki/Omnidirectional_treadmill

The Virtuix Omni is a device like what you have in mind. Essentially you run in a shallow bowl shape with slippery foot pads and it holds you still at the waist.

http://www.virtuix.com/

https://www.reddit.com/r/oculus/comments/49nbih/a_full_month...

The problem with this type of solution is that you still don't feel the acceleration, you're just flailing your limbs around in a single spot without actually moving.

It's the missing acceleration that makes you sick, not that you can't move your limbs.

Not having used it - I would think of it like an omni-directional treadmill. Treadmills don't make me feel sick, why would this? Legitimate question.

The sickness comes from the disconnect between what you see and what you feel.

When you're running on a treadmill you see that you're standing (relatively) still, just moving your limbs. Your inner ear corroborates this when it doesn't feel any horizontal acceleration. Everything agrees; no sickness.

In VR, you see that you were stopped and then the gun went off and you started running. But your inner ear disagrees: you're not accelerating at all, you're still standing still. Disconnect, sickness.

Just stick some electrodes on your head to poke at your vestibular system so that it matches up with what you're seeing

http://www.fastcompany.com/3058414/mayo-clinic-technology-sa...

If their system actually works hopefully we'll see consumer implementations in the next couple of years.

I like it, it's even compatible with my little GearVR. Of course, if I had $700+ to spend on peripherals, I probably wouldn't be using the budget headset in the first place.

What if you rotate the camera slightly while you move? By the time you reach the end of the physical room, the camera is facing the opposite direction in virtual room and you have to turn around.

It would be worth a try. My bet is that the user vomits or falls down.

They've done similar things along with other tricks, like changing the layout of the room when your back is turned to simulate large spaces in small rooms.

http://www.wired.com/2015/08/cant-walk-straight-lineand-that...

For 'redirected walking', you need a lot of room because you can only bend a little. The Void has done it and researchers have, but they usually have warehouse-like spaces. It won't work in a normal 3m^2 sort of residential room, which is why you need gimmicks like teleporting or omni-treadmills.

if it's small enough that user won't notice, you'd still need a big open space. Otherwise it's obvious and obnoxious. There was some games where once you got close to a wall, you could spin in place and the world would keep up with you, but this means you can't have any kind of fight or escape scenes in your game.

It might be obvious, but the user could probably get used to it and learn to live with it.

For the other idea, the game could pause during rotations. Would be annoying, but it wouldn't destroy the game mechanics.

No solution is going to be perfect. I think people interested in this will probably have to do it outdoors, or at some public space.

Just that, say you play one of the survival horror games that people are waiting for to hit VR, you get jump scared and have to run away from the monster and you just blow straigh into a wall. unless of course it starts to show the turning thing way before you a wall (like 2 meters away) which would mean you have to have big room to play in.

Reply on news.ycombinator.com