DensePose: Dense Human Pose Estimation in the Wild | Hacker News Reader

DensePose: Dense Human Pose Estimation in the Wild

densepose.org

114 pointslainon17 comments

Question: how far away is this from a "person swapping" GAN video synthesis kind of application?

I’m not sure if you’ve seen the face swapping that was making the rounds recently, but I was thinking this makes whole body swapping look closer.

That's exactly what I had in mind, but I'm no expert. And just wait until they've got basic, fluid object recognition down. I've already seen some impressive, old work on coherent in-painting of images.

It seems like all these deep learning discoveries are relatively composable too.

If you thought face swapping was bad, just wait until someone uses this tech to create AR goggles that let the wearer see everyone “naked”, like having Superman X-ray vision

If the software is at all accurate, using that kind of AR google on me should result in retinal scarring.

Why would that be 'bad'? If someone finds joy in seeing all people naked, go for it.

I think that's a really good question. IMO this is an essential part of any pipeline aiming to do that, but there's a bunch of hurdles with neural networks that's haven't been crossed yet. The first one that comes to mind is that I'm not aware of any work which would try to teach neural networks to imagine what someone would look like from different angles (aside from relatively restricted settings like facial frontalization). Second, you talk of videos. That means the person needs to look 'consistent' from frame to frame. This isn't a solved problem either, especially when it comes to GANs. I'm sure there are a few other areas that need to be solved.

I think 'person swapping' would be best done end-to-end. No need to go through something like this.

This looks so cool! I wonder if they'll make training data available

Seems like it: "The dataset will soon be available on this website!"

This is awesome! Will the code eventually be available on detectron?

Hi, could someone please pretend I am an HR rep and explain this to me in simple terms?

This could be handy for vr?

And AR! Though I'm guessing it will be pretty hard to get it accurate enough for VR.

I would love to see a version of this applied to both VR and AR but it seems that the current implementation is still slow for the performance that VR requires.

From the site: "During inference, our system operates at 25fps on 320x240 images and 4-5fps on 800x1100 images using a GTX1080 graphics card."

For good VR/AR experiences you'll want better FPS

You don't necessarily need better FPS from the pose tracking, because you can use optical flow to interpolate and predict movement. These sort of techniques are already commonly used for VR positional tracking off camera input.

Amazing!

Reply on news.ycombinator.com