madhawav

Oh yes. This could be a nice interface to provide other inputs we (plan2scene) want as well. Like photo to room assignments. Then, plan2scene can create 3d houses with appropriate materials. Thank you for sharing this link.

Do you think its possible to decouple lighting effects from the generated 3D scene? Currently it seem to simply copy the actual lighting condition in the environment. In a way, it's a limitation. It's much better if it can provide the RGB color + other material information (SVBRDF decomposition). If that's possible, then these meshes can be used in other environments.

This is actually a very interesting video. Reflection effects look very realistic.

I did got a delivery failure on one of the emails I sent out. Can you submit again just to be sure?

Here is the form: https://docs.google.com/forms/d/e/1FAIpQLSfl4muDFf0qktqtWhj6...

Sorry for the trouble.

Did a bit of googling and this came up: https://canvas.io/. But, it's not for Android.

There are some prior work that can digitize (vectorize) floorplans: https://github.com/art-programmer/FloorplanTransformation.

This can detect rooms as polygons, doors/windows as line segments and bounding boxes around objects (e.g. cabinets, sinks etc. indicated on floorplans).

For instance, look at this sample 3d scanned house: https://matterport.com/en-gb/media/2486. Make sure to switch to the doll-house view from the toolbar. There are many holes in the scan.

Thank you.

This is definitely a thought I have considered. It could potentially ease lot of hassle related to 3D scanning of houses by real-estate companies. Very often, those scans require expensive equipment, professional expertise to use those equipment and considerable time and effort to capture the entire residence. Even then,you will find many holes in those scans and floating debris. With our work, one only needs a sketch of the floorplan and photos taken from that house. Plan2Scene will use data to infer appearance of unseen surfaces.

Thank you!

Estimating the global camera pose using non-overlapping photos is a challenging problem. This is doable if the photos densely cover the house (e.g. video frames).

However, if the global camera poses are available, we can detect surfaces in photos and project them reasonably well into 3D. In the past, I have used this work to achieve something similar: https://github.com/NVlabs/planercnn

Hi, This is because the Rent3D dataset we extend has a "non-commercial use only" term.

I'm the first author of this, happy to answer any questions. It's great to see this on HN.

PS. I'll be graduating this July with my masters.