Nah, AI will calibrate and then calculate your bone movement by measuring tiny vibrations through earphones. Every moving part of you body could be measurable that way.
And why would you move your hands if you don't directly interact with objects, you could track your tongue touching or sliding along your teeth, best interface ever.