DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation | Hacker News Reader

DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation

yaroslav.ganin.net

104 pointsDowwie26 comments

Badly needed for video calls! Real time, please.

What I do is minimize the remote video and move it to the corner of the screen, as close to the camera as possible. If the other person does the same, we are then close to making eye contact while gazing at each other's images.

Your conversation partner reads out a passage from a book... without looking at it. Instead, his unwavering, slightly deformed eyes stay fixed upon you. Always.

Of course this would be avoidable through careful implementation: only redirect the gaze if it's already directed at the screen, or even specifically at the correct part of the screen. This might require a calibration step, or at the very least, it would require the software to be aware of the relative configuration between the camera and the screen.

Still, personally I'm not really comfortable with any software systematically editing the direction of my gaze.

Neat. Although in most of the examples I looked at, the NN seems to get the shape of the eyes uncannily wrong. I would have thought that would be the easy part. I haven't looked at the paper.

I thought that this was going to be about 'Gaze Manipulation' in the sense of manipulating which part of an image draws your attention, e.g. [0]

[0] https://sites.google.com/a/g.rit.edu/gaze-manipulation/

I think it wouldn't look so goofy it wasn't animated. If you saw a still frame, you probably wouldn't notice.

I feel like there's an inordinate amount of effort going into making funnier and funnier gifs these days.

Yes, all of the gifs are literally comical. If instead the gaze had followed the cursor (very commonly done, first example I found: http://www.javascriptkit.com/script/script2/xeye.shtml ) it could have been a lot more amazing. it loses its seriousness.

OpenWindows had a demo called "monaeyes" that made it seem like Mona Lisa was really interested in whatever your cursor was pointing at.

It's kinda unsettling that the lower eyelids never seem to move. It makes it feel more obvious that it's a graphical trick, rather than natural face movement.

Reminds me of "Locally Smooth Manifold Learning" [1], specifically figure 6 in "Learning to Traverse Image Manifolds" [2].

[1] http://vision.ucsd.edu/project/locally-smooth-manifold-learn...

[2] http://vision.ucsd.edu/sites/default/files/DollarRabaudBelon...

That googly-eyed bald guy is, quite literally, the worst thing ever. I'm not sure why, but it was deeply disturbing in a very strange, uncanny-valley sort of way. Perhaps it's the fact that they've not got the synthesis _quite_ right yet, and something is tapping on your subconscious' shoulder going 'that's not real' while you are happily looking at it, and seeing something that - to all intents and purposes - is an actual, real video of a person?

He's a celebrity on Reddit, his name is Harold.

https://www.reddit.com/r/youdontsurf/search?q=harold&restric...

I wonder what the effect would be if instead of these large but regular movements, a very small random movement was applied. My guess is that it would make pictures more "alive" in an uncanny way without obviously distorting them or making them funny.

Very cool. I imagine this is targeted at video calling.

No need to imagine, that is literally the first usage scenario mentioned in the paper.

The animations reminded me of Terry Gilliam.

This approach seems to ignore the fairly major role the eyebrows play in looking in a particular direction.

Is there an open source code implementation of this? This looks like a lot of fun to mess around with. :)

http://yaroslav.ganin.net/static/deepwarp/files/gifs/merkel_...

Very cool. In this case, however, is the deep learning approach gaining much over a traditional custom coded technique? There seem to be a few such papers in the past few years.

Google Chrome prevents you from visiting this site right now. Bypassing it's warnings leads you into a compromised site.

Should be fine now. Someone screwed up the DNS records.

How does it perform on a face even slightly in motion?

This is amazing tech. Super creepy though.

Missed opportunity to give them all derpy eyes.

snapchat acquisition in 10... 9...

Creepy.

Reply on news.ycombinator.com