The moving physical objects makes sense, touch makes sense. Learning a series of gestures to manipulate data on a screen makes less sense to me than a mouse.

The mouse is movement and control, the gestures need you to hold your hands in strange positions and move within a field of view (what the camera can see). An improved design will need to be intuitive.