I think that moving a finger to do an action is going to trump needing to move your whole arm or do jumping jacks for the same action. Especially since it would be fairly embarrassing for probably a whole generation of people before it became common place. But it might justify some of the more active users getting offices.
What you're suggesting I imagine is points at an item but doesn't touch it. Maybe they coordinate themselves with a virtual reflection on the screen or something so they needn't physically touch it?
The background technology is indeed complex, but that doesn't really matter much. The biggest stopper is the fact that you are still putting and abstraction between yourself and the object you're manipulating.
Touching something and directly manipulating (touch screens) is a very close coupling with little abstraction. But having to point, but coordinate your "reflection", or point but don't touch is a stretch.