PIFuHD: High-Resolution 3D Human Digitization
shunsukesaito.github.io
shunsukesaito.github.io
Great research coming out of Facebook Reality Labs!
I can nearly guarantee you that if you try to to use an image with something on the back it won’t be able to fit it, backpacks don’t work and probably something like the Victoria’s Secret angel wings won’t either.
You can also see where it really fudges things like the shoes on some of their examples, their examples are also the best case scenario (similar to the StyleGAN2 results) and not necessarily representative of what the model achieves regularly.
I’ve tried using some pictures of my partner in order to 3D print her but it didn’t really work well I’m guessing you need a really clean background.
This guy on Twitter did some of his own tests and they came out pretty good. https://mobile.twitter.com/Yokohara_h/status/127239671259470...
“We demonstrate that our approach significantly outperforms existing state-of-the-art techniques on single image human shape reconstruction by fully leveraging 1k-resolution input images.”
edit: downloading now, maybe the server was overloaded.
Backpacks, and other attachments that are on the back or occluded ofc can’t be predicted or fitted with-in the model to represent their real life counterparts well not unless you train the model on every possible backpack.
That said training this model on people’s photographs and their 3D scans and then possibly on a dataset of clothes and accessories from big brands might actually allow you to much more accurately predict the unseen parts of the image by not only filling the gaps based on predictions but by actually specifically matching the items of clothing and accessories to draw additional information that way.
Most fashion brands already have detailed images of their products from multiple angles and even have 3D views of many items.
Combine that dataset with this model and you’ll probably be able to increase the accuracy of these models considerably.