Hacker News Reader

Comment by potlee | Hacker News Reader

Can someone who understands this tell us what is the catch here?

You'll probably hit memory limits if you go much beyond 512x512-sized holes.

Additionally, computation times grow quickly with higher resolution and you already need a high end GPU for this resolution to get a reasonably interactive response time.

You'll also need a favorably licensed pretrained model or a few 10000 training images and masks.

So all in all, I can't see any deal breakers, but I'd probably still use PatchMatch instead.

For reference the GPU they're using for this paper is the NVIDIA V100 GPU, a datacenter GPU costing $8,000.

To be fair, while V100 perform very very well for machine learning, you can buy almost a dozen 1080ti's or a few titans (whatever the current one is), which would certainly be much faster.

They say they used V100 but not how many, if they needed a large number then nevermind.

The paper says they only ran it on a single V100, I was expecting multiple GPUs as well.

Since they programmatically generate the masks you wouldn't need those, just the set of training images. So it wouldn't be too hard to find since you're not looking for paired images, just a bunch of images of faces/landscapes/whatever you're trying to inpaint.

The catch is it is more like an artists interpretation. Anyone expecting that it will truly fix old damaged pictures to be like they were is going to be disappointed. Anyone just wanting to fill in some gaps will be excited. (Same for just procedural generation of some images, I suppose. Would be neat to see what it can do from a very low fidelity outline of a house/forest/etc.)

As opposed to it being truly magical, I guess?

I think the expectation has to be set that this is great for creative use cases. Not so much for forensic style ones.

To that end, "reconstruct" is not so much reconstructing what was lost, but more "fills in holes" of photos. If you read the title expecting something to literally be reconstructed, you are likely to feel lied to.

Reply on news.ycombinator.com