I'd say it's primarily caused by a disconnect between what the human vestibular system (your sense of balance and spatial orientation) is telling the brain your body is doing vs the acceleration forces that your body is feeling vs what your eyes are telling you about what the body is doing.
Or in other words - differences in movement/locomotion your eyes are seeing in HMD vs what your other senses are experiencing.
Which probably would include latency as a subset.
To note, simulator sickness is not that new, we have been grappling with this for ~65 years since the first flight simulators. Despite the massive funding that the DoD has at it's fingertips, we have not found a cure for it OR there has not been a lot of work done to find a cure. However, it seems that more time in the simulator does help, thought it may then hinder actual flight performance. Also, the more experienced pilots had a higher occurrence of it. Again, it's not that well understood.
It's a major hurdle to overcome.
https://en.wikipedia.org/wiki/M%C3%A9ni%C3%A8re%27s_disease#...
Still, if it were a total scam I wouldn't think the Mayo Clinic would put their name on it.
The vestibular system has a prodigious amount of control over your life. I've got hearing damage and occasionally get vertigo spells (I carry dramamine in my wallet for emergencies). Every once in a while my balance sense just begins to swing around wildly. Sounds kind of cute. It will put you on your ass for 24-48 hours with zero possibility of doing anything. Even drinking water is hard. Without drugs the nausea just becomes your entire world. You think about nothing but how much it sucks. You can't eat, move, sleep or speak.
So... yeah. Motion sickness is a very real thing. I think (based on zero evidence) that everybody has a different threshold but experiences the same effects past that. If you're lucky, your threshold is faster than you can move your head. If you aren't, chug dramamine and coffee and hope they invent something that turns off your sense of balance without making you delirious.
It's a pretty demanding latency test, and I can also see how it ties into balance etc (there are some similarities with a tilting boat deck, for example).
That said, I think simulator sickness and sickness from just playing on a big screen is complicated - maybe some will never enjoy it.
Personally I'm rather sensitive to motion sickness from most fps games - but I enjoy playing elite:dangerous with a vr headset.
Rollercoasters and smooth/fps style movement is rarely used - and even if it is, the interface is designed tomminimise the effect you mentioned - e.g. user is put into a virtual cockpit, which makes the brain feel it's the terrain that's moving and not the person.
Are you responding to someone? Because this doesn't really make sense to me. Can you clarify?
Maybe it's a due to a lifetime of playing video games (3d since i was 10), but even VR games with weird detached cameras or games not designed for VR which don't really respond to head movement are totally fine for me. There is a cognitive adjustment I make which is manageable and does not make me feel sick.
Having used gearVR with a scratched phone screen for a year and now real Oculus for a few months, I am fairly certain that my particular sickness comes from the strain of my eyes constantly focusing too close or trying to mentally blur obvious pixel artifacts or struggling to focus on fogged up parts of the lens which is causing me to experience a headache which eventually leads to nausea.