Depends on the image. Right now, the very first stage (deprojecting the image to 3D) makes assumptions about the image having the structure of a room: large empty floor plan; roughly polygonal geometry.

For different kinds of images, it's a question of using other cues to build a 3D structure that's very close to the original image. And no, monocular depth estimation isn't enough (happy to nerd out about why) ;)