Hacker News Reader

Comment by btown | Hacker News Reader

To quote the paper:

> Our model is supervised with paired input and target views of a scene (along with their camera poses)... The model then needs just a single image at test time.

Correct me if I’m wrong, but: Given a novel scene, it seems the model must be retrained on multiple images of that scene? It seems disingenuous, then, to say it works from a single image. No doubt the interpolation is state of the art, but the title seems misleadingly magical.

View on HN

I think you're misunderstanding the paper. They are using multiple images in the learning phase to infer the 3d structure. At test time, it works on novel scenes with just a single image.

I believe from a second reading this is correct - after training on the Matterport real estate image dataset, it appears they can transform novel images (which happen to be from that dataset, so they are still home interiors but can handle completely distinct scenes with distinct furniture etc.). Which is actually really fascinating and leverages decades of innovation. This is actually really cool and fully justifies the hype in the title.

I don't see how that would work with the paintings.

Very curious to understand this as well. Does that imply that images with similar attributes (like other hotel rooms in one of their examples with slightly different dimensions or furniture) will work without additional training? Or did each example require multiple image training of that scene before a 3d mesh can be computed from arbitrary angle? In the latter case it can still be useful as compared to photogrammetry which would require hundreds of photos to achieve similar results, so maybe this could work with a dozen or less?

Unfortunately that's most(all) of academia!

Reply on news.ycombinator.com