Suggestion: Use "photosynth" techniques to fuse the video frames from the moving cameras into a 3D model of the room.
EDIT: When I read photosynth, I was thinking of Microsoft Photosynth[1], which sketches pictures into panoramas. I should have read the parent comment a little more closely.
Also, since the camera is moving, one doesn't need to use multiple.