In Virtual Reality, How Much Body Do You Need?
nytimes.com
nytimes.com
The design space for HMD and body tracking UIs is much larger than skeuomorphic VR. I've liked attaching eyeballs to my hand, to explore graphs without gymnastics or manipulation. Using aphysical kind-of-planar workspaces, to permit subpixel resolution on integrated graphics. Remapping motion, to permit fast ergonomic input from resting hands.
Non-skeuomorphic UIs aren't getting much attention yet. And even awareness of that is not yet widespread. But there's a lot of fun incoming.
But i agree that, VR-design wise, we are very primitive right now.
Yeah, this is the common practice.
But by "attaching eyeballs to your hand" i though he meant decoupling the camera from HMD tracking and attaching it to the controller tracking. Like you move your hand and the view changes. Something similar to The Pale Man from Pan's Labyrinth [1].
Research shows[2,3], that brain adapts to the unusual perspective quite fast. So technically, after a while, the brain should accommodate and it would start feeling "natural".
[1] https://vignette.wikia.nocookie.net/slendermanconnection/ima...
[2] http://www.yorku.ca/harris/pubs/adria_inducing_preprint.pdf
Yes and no. There seems a lot of confusion about that. About which design constraints are coming from which goals. Yes, for the common style of immersive gaming. But when I'm working, I'm usually down around 30 fps with high variance, running on integrated graphics.
What constitutes a "horrible" "immersion-breaking" "visual artifact"? And how much will you pay to avoid it?
I'm looking at a laptop's desktop. It's obviously a display panel. And not my wooden desk. And that's fine. When looking at my HMD "desktop", it's also obviously a panel. And not my office. And that's also fine. Using emacs isn't fighting slimey zombies. Usually.
Paper novels can be immersive. Even if you sometimes notice turning a page.
The design goal "avoid reminding the user they're wearing an HMD" is a very challenging one. It prunes the design space, discouraging many things. Camera passthrough AR. Lag. Different objects having different lag. Visible boundaries in display space. And so on.
For example. If you mostly care about text, then you want to see unblurred pixels, which means only the center half of the lens-blurred display is useful. Passthrough AR can provide balance even at low fps. So it can be shown beyond the center. Which creates a visible boundary in display space. And that's fine. And if when you turn your head, some graphical elements slowly chase others across the screen, that's fine too. So now you can run on old Intel integrated graphics. It's a different point in design space from fighting zombies in a warehouse. And it has different design constraints.
So when people say "VR requires X", that's worth translating as "SteamVR games require X". And it can be fun to consider other things you can do with a tracked head mounted display.
There is a difference between design decisions based on "striving to achieve presence" and design decisions based on "not make people vomit". And high fps with good tracking is closer to "not make people vomit" group.
I agree that it's okay to break presence to do something (i would trade presence for fun for example) and let user push the limit if he can handle it, but you have to understand that VR is already a niche market, so devs usually are just playing it safe.
A goal of avoiding visible lag, requires a mechanism of predictive tracking, which has an undesired side-effect of judder, which can be far more sickening than the original lag, which motivates a mitigation of higher fps and lower variance, which requires a stronger GPU.
A goal of wide fov (absence of tunnel-vision "comfort mode"), increases user sensitivity, especially during rapid head and spatial motion, increasing needed performance and user discomfort risk.
A goal of avoiding visual artifacts, discourages introducing visual artifacts which increase user comfort. A goal of visual seamlessness, discourages segmenting the display to permit separately optimizing for task and comfort, compromising both, motivating mitigation.
With current hardware, "presence" and comfort are conflicting objectives. Optimizing for presence, is why we're pushing the envelope on comfort. But there's been a lack of community awareness of just what tradeoffs are being made, and why, and of alternatives and opportunities. In part because of the niche economics you mentioned - if SteamVR doesn't support it, why think about it? But happily, improving hardware will make the issue mostly go away.
I am convinced people who talk about immersion in VR have never been addicted to any game. It doesn't take that much to lose yourself inside of one. After the first few minutes both VR and non-VR games are the same once you're fluent with the controls.
Again proving the point that VR has no actual use case.
I'm not an ayatollah of VR, but without any foundation to support it this is a pedantic and ignorant comment
So VR is definitely not pointless. Motion tracking for your head/arms/feet unlock many applications that are previously hard or impossible (e.g., 3D sculpting using mouse/tablet vs a motion controller).
But it's not always better than mouse/keyboard input for let's say typing or shooting a target. Having more irrelevant degrees of freedom makes it harder to master your input controllers, even if you already had a lifetime of practice with it in the real world (e.g., I am okay with doing parkour levels of movement with WASD/Space but won't ever try it with my real limbs even though in theory I know how to climb over a real world fence).
FEAR is a game that really underscored this to me - and every game I see it in afterwards just adds evidence for it.
Required? No. But adds to immersion a LOT.
Given that you had to enter pin codes by setting the hand to pointing mode and move around with the mouse, i wonder how it would translate to a VR experience today.
This couldn't be farther from the truth.
Take a game as simple as Zombie Training Simulator. Holding your hands up and firing virtual guns is a completely different experience than pointing a mouse and clicking. Physically bending over to pick up a grenade, then throwing it, is far more immersive than pressing the number 5 and holding a mouse button down to prime it, then releasing it to throw.
Then there's Ultrawings. Having head tracking, being able to look around by just pivoting your head, makes the flying experience feel so much better, even if the graphics are dated by 10 years in terms of texture and model detail.
When I got my Vive and was playing around in The Lab, and one of the experiences puts you on the side of a mountain, I stepped off a cliff edge and I could feel my heart rate increase slightly as my brain was expecting me to fall to my death. The immersion is real.
And I have plenty of gaming experience. I've been gaming since I got an NES for Christmas when I was 5 in 1987.
Not disputing anything you're saying but I actually have this same experience playing games where you can fall and the player accelerates (particularly if the camera is first person or right behind the player). Just looking at the screen when I jump off a cliff in something like WoW makes my stomach feel like 20lbs of iron while I'm falling. I've kinda learned to enjoy it at this point.
What's interesting is that after far too many hours of WoW, that fall distance is so ingrained that I get that feeling at the same point in other games, even if they have a longer 'safe' fall distance.
In the end, I ended up making a tiny tiny step and leaning forward enough that the game registered me as having jumped. The two experiences aren't even comparable. In one game where I was in a hot air balloon I even freaked out and panickedly searched for the button to exit the game.
Or closing their eyes. Something they would do for a movie when there is an uncomfortable segment.
For example, the distinction between being deeply involved and distracted in a game (which does not require special tools) versus the extent to which the game will trigger automatic bodily reactions.
In other words, someone deeply "immersed" in a chess-match isn't quite the same as someone "immersed" in a VRv game so that they fall over at a jump-scare.
I imagine with practice anything is possible regarding VR. The mind can make anything believable if you just relax and let it happen.
That's generally good advice for journeying of any kind, whether we're talking about VR, psychedelics, drumming, meditation, lucid dreaming, or reading a really good book.
I am learning VR Development in my free time just to explore this. I want to make a VR guided meditation program, to help beginning mediators reach deeper states than they would normally be able to.
Having spent many hours in both vive and rift, while I prefer vive in general, the visible hands on the touch controllers on rift is just fantastic.
A laser-hatchet hand and a disembodied floating mecha-claw that respond snappily are ultimately more real than a "realistic" arm that moves like a cartoon.
For yourself you do not want to show body parts that cannot be tracked correctly because of the jarring effect of the mismatch, as you say. But for showing other people you can get away with more, so for example you can show full arms even if the elbows are not exactly in the right place (as long as they are not unphysical of course.)
> I remember trying a vive game that had arms but sometimes they would contort into impossible poses and it was extremely jarring [.....] the visible hands on the touch controllers on rift is just fantastic.
Neither of the things mentioned here are specific to Vive or Rift.
A Rift game can give you arms that contort into impossible poses, just like a Vive game can give you visible hands. It's just software and whatever the author of whatever game you're playing decided to do.
Did you play a game worked in both Vive and Rift, and the versions were different or something?
1: https://upload.wikimedia.org/wikipedia/commons/d/d8/Oculus-R...
First thing I did when I played a demo of Dark Secret, it shows your character in a mirror as a way to show you what your character looks like (there's the mask mirror from the Oculus Dreamdeck as well). Of course I see the character's head tilt to my head movement but most people when looking at a mirror make an expression (smile etc...) and the fact that it didn't show my actual facial expression reflect in the mirror was off putting.
Second experience was Job Simulator. I picked up something, dropped it on the floor. Tried to kick it away with my feet but of course no sensors on feet so that failed. Also tried to hip close drawers and file cabinets but of course no hip sensor.
If I was in a VR meetup I'd want to be able to express with my hands meaning I need all 5 fingers so I can point, give the middle finger, make the OK gesture, pick up a glass and stick my small finger out, make the shaka sign, etc.
And a teleporter https://sean-bradley.github.io/TeleportVR/
I think if we had some other way of controlling your avatar/viewpoint, such as some kind of direct mind interface I think you could put people into nearly any kind of body at all and they would adapt.