Amazing 3d iPhone app
kode80.com
kode80.com
If you watched the video, you've pretty much seen the extent of the app.
The main barriers I faced was the processing power required to monitor the user's position accurately - needing a camera with relatively good resolution (to get accurate depth perception) and good frame rate (to reduce latency). Also, what to do when another person enters the scene? (Keep tracking the primary / previous target)
Also, using it as a 3d desktop / window manager is probably not worth that much drag on your system - but I could see it being quite useful for cad systems / 3d modelling / prototype development.
Also, developing this way could lead to some interesting developments in the UI field - there's already some development here for the disabled, so is not entirely new - but hasn't really made it to the mainstream desktop...
http://tinycartridge.com/post/413663037/dsi-motion-tracking-...
I believe they use the built in camera to sense motion, as there's no accelerometer in the DSi.
There are only really 3 models/rooms so it is only really good for showing off to other people.
http://www.youtube.com/watch?v=I8U6ik6ryQo
I suspect this got removed from the app store in the recent smut purge, though.