‘Deep Nostalgia’ can turn old photos of your relatives into moving videos | Hacker News Reader

‘Deep Nostalgia’ can turn old photos of your relatives into moving videos

gizmodo.com

Edit: Seems like its based off of something similar to this (https://github.com/AliaksandrSiarohin/first-order-model)

This is pretty cool and eerie at the same time. The biggest thing to keep into mind about these algorithms is that the computer “dream” of these moving videos so they can be inaccurate. Nevertheless, seeing some moving video of my ancestors would be pretty neat.

Within about an hour, I got sent the link to Deep Nostalgia by several acquaintances. None of whom are AI Researchers or particularly tech minded. Daily Prophet come to life resonates. ML techniques that touch universal, ancestral themes clearly have more mainstream appeal than an agent that can solve geometry proofs. Raw human emotional response is thus a crucial component of evaluating Beneficial AI.

Alas, the image you see in your head is also a dream created by your brain's neural networks.

It's a dream created using information from many other scenes you saw of the person.

The AI "dream" is based only on the single image.

I’m not an expert but my understanding of it is that our brains aren’t like computers. So, rather than storing precise memories, we probably have multiple memories sharing the same neural connections. Like, the memory of grandpa’s beard is probably mixed up with the memory of all sorts of beards.

In that way, the artificial neural net seems similar (while at the same time vastly different).

Not if you have aphantasia.

https://en.wikipedia.org/wiki/Aphantasia

Even people with aphantasia see as a result of a combination of bottom-up and top-down processing.

Also note that aphantasia is not a condition which is confirmed to even exist.

As an aphant I’m pretty sure it exists. But just in case let me close my eyes and double-check...nope still nothing there. ;-).

I'm pretty sure that you either see the same thing as I do when you close your eyes, or you don't, and that it's very difficult to communicate about what "seeing" something in your mind actually means.

Except brain's neural network don't work the same way

My understanding (happy to be proven incorrect) is that's still a matter of disagreement. AFAIK, the idea of the brain having feature detectors of various kinds remains a matter of research as it has been for decades.

You don't need to answer the general question. The brain can't be reasonably modeled by this particular neural network.

Honestly it reminds me of an effect straight out of Harry Potter films. Would be an interesting affect if it could be easily applied to otherwise static billboards and such

s/can be inaccurate/are a complete fabrication

I think that puppeteering the dead is actually quite a disturbing thing to do in general, and find endeavors such as the "resurrection" of Tupac to be in very poor taste.

It’s a fair point but I think there’s a difference between some music group that owns the rights to Tupac’s likeness creating an image of him to sell more things and a family member wanting to see an deceased relative in motion.

But I take your point that it could be used in distasteful ways, much like how Tupac was used (IMO).

It can be both a fabrication and accurate, though. That's what people are so impressed by here.

But it can't be accurate. Actors know that even micromovements tell a story and convey emotion. The computer cannot possibly infer enough about the person from a single photo to reconstruct that story. Were they a fidgety person? Were they tired that day? Bored? Impatient because they need to get the store before it closes and the photoshoot is taking a long time?

Imagine running this software on a photo of your parents. Do you think you would be impressed by the accuracy?

Could not agree more. I also maintain the unpopular opinion that speculatory colorization of black and white historical imagery is also doing damage to the accuracy of the general historical canon.

Accurate to what? How some other person moves? Its like wearing someone else's skin Ed Gein style. Maybe if they were modeling it based on other photography.

> seeing some moving video of my ancestors would be pretty neat

I've tested it and it only really works with high quality pics taken against a plain background. Otherwise your ancestors look like ghouls.

This would also work for a scary room in amusement parks. Imagine Harry Potter-like photos of yourself + previous visitors screaming and trying to get out of the picture frame.

Check out my five minute old son looking around. Really weird.

https://myhr.tg/1O7m77RK

And these articulated sunglasses.

https://myhr.tg/1rX4Z8Xs

John Quincy Adams came out a bit off:

https://i.imgur.com/Bn0AgkJ.mp4

Straight-on photos seem to work a lot better, for obvious reasons. It's not very good at compensating proportions for turned heads.

A friend of mine, her mother from eastern europe in the 1970s:

https://i.imgur.com/iLc4iIt.mp4

That one came out decently.

The newborn photo is especially funny because newborns don't have the requisite neck strength to be able to look around like that.

Interesting that the AI understands sunglasses but still "blinks" them like eyes.

Is it really artificial intelligence and not facial recognition plus a series of image transforms

It's not just an image transform, because it has to create facial features that were occluded in the original. (And correctly occluded existing features when the head turns.)

In the video with the sunglasses, for instance, the computer had to invent that ear on our left. (With mixed results -- it looks a bit like Mike Tyson got at it.)

There is something so wrong with that! But your son is VERY cute and not creepy at all!

I would be just like Harry Potter, when it would be possible to print moving photos on a paper or a wall. With a sound preferably. I liked the idea of an moving image of Harry Potter repeatedly saying "Eat dung, Umbridge". Wouldn't it great to paint a graffiti saying obscene things to a passers-by? Then you could hide behind bushes and record a video how people react. Oh, and you could draw another graffiti which might answer to the first one. The city would become a different place, I'd stop using internet and start using reality again. It would be so much fun. Not for a long I expect, then morally wounded citizens would ban moving graffiti to make world boring again. But for a few years we could have a lot of fun. And then move to underground.

More likely the technology you envision would be applied to packaging and would turn supermarkets into an even more unbearable kaleidoscope.

You need programmable physical paint

imagine going to a house of horrors and there's a painting of a dead relative

So freaking amazing how good the 3D depth perception is on that 3rd GIF. The right side of his hair is correctly obscured when he turns his head and even his eyelids move correctly. The only slight issue is his "lazy eye", but then you could argue the AI took a little creative liberty

I can't be the only one who has thought that deep fakes + Cameo = moneymaker! Pitch the celebs "hey you can have 70% of the cameo money, but you don't have to lower yourself to _actually doing the cameo_". We'll just take the other 30% to pay actors to do it for you.

Thought about it? There was a movie made along those lines:

https://www.imdb.com/title/tt1821641/

If you like bad movies, this one manages to be bad on a number of interesting dimensions at the same time. Highly recommended!

Having read the book, I'm just amazed anyone could have read it and go "we should make a movie out of this!" And I love all of Lem's writing.

I watched the movie and had to read the wiki page about the novel to follow it. And I'm a) no longer smoking weed, b) used to teach a history of film class in a lit department at a university, b) also enjoyed Lem when I was reading those works.

I enjoyed the film, but it's very difficult to read without at least a glance at a synopsis of the source material.

The Prior-Units theorem states that if you can put an idea into words, there exists prior units in proportion to the utility of the idea (so there's an infinity of novel but useless ideas).

Is there a variant of this along those lines but in regards to "someone made a movie about this"?

'Simpsons did it' is as close as I've got.

Haha I was thinking that too. Maybe that's the best we have.

More interesting is the book it's loosely based on, where protagonist is split between delusional and real mental states. Stanislav Lem's The Futurological Congress, excellent read.

The movie 'Cam' is actually pretty good, but its concept is that the AI takes 100% and pays the actor 0%.

https://en.wikipedia.org/wiki/Cam_(film)

The thing is, you need to find an actor who can do a perfect impression of your rhythms and mannerisms -- facial, bodily, and vocal.

Which is actually incredibly hard. It's easy to do jokingly and exaggeratedly for SNL. But getting it down enough to be believable is a ton of work.

Also, celebs LOVE doing cameos. They're not lowering themselves. It's a blast to do. Except in very rare circumstances, celebs aren't doing cameos for the money.

I imagine when processing power doubles a few more times that we'll be able to do that in real time. Pick your movie, then pick which actors you want, use your Mii as the next James Bond, etc..

It would be in Cameo's interest to ban deepfakes preemptively (since devaluing actors, their primary content creators, would not be wise even moreso if they are unionized), and I'm honestly surprised they haven't already.

https://mashable.com/article/damian-lillard-hulu-deepfake-co...

I see the next wave of copyright suits coming already:

> 30-odd geek sued for Billions for infringing on the "Brad Pitt" brand.

Interesting times...

So if I happen to look like Brad Pitt (I don't), then I can't have a role in a movie because of his "brand"?

You can't try to imitate him to trick audiences into thinking it's him, no. That's already been a lawsuit (well, not Brad Pitt), and resulted in SAG rules against it.

https://www.syfy.com/syfywire/revealed_the_real_life_dr

I don't think this would take off. Part of the fun of cameo is having celebrities take the time to recognize you.

Technically this is amazing but I find it too disturbing as a thing. This is like creeping out the past into something it was not. Again, I applaud the technical aspect of this and it was bound to happen sooner or later. Will it become a thing where we take our long dead great great grandparents for a ride? Im not sure but I hope it will not

Yeah, I just feel deeply uneasy about this and I can't really put my finger on why, perhaps it just feels too flippant. My sister died 20 years ago and i feel like if i were to upload her image I would run the risk of overwriting my own memories of her somehow? I'm sure others feel differently but this isn't for me.

It's exceptionally creepy. I think you nailed part of the reason why: it's an imposter, it's fraudulent. Straight out of the twilight zone.

Just wait until AI malware starts impersonating people we know or used to know in a concerted harrassment or manipulation campaign. Oh the evil shit people are going to do in the next decade or two.

I used this to animate a relative of mine that we had just discovered in court documents (from a mugshot from 1926) and blew my family's collective mind. I had to keep reminding them that this wasn't real - this is an AI-driven fantasy. They did not seem capable of understanding what I was saying. I am scared of our Black Mirror future.

Looking at the examples shared here I thought "that's pretty cool, and would be amazing to try, doesn't look creepy at all".

I loaded a portrait of my grandmother taken 60 years ago and it's weird, uncanny valley and very very creepy to see.

It's odd to see a photo you've seen many times of a person you knew so we'll come to life. It's the things that are missing, the small expressions, the details in the way they move the facial muscles, the "life in the eyes". I'm sure if it's of some long forgotten relative it would be interesting to bring a photo to life but not for someone you know.

Interesting. It gives one plausible animation, but it is one of many plausible animations. So it looks good for people whom you have never met before, because it is plausible. Yet it is not as good if you know the person, because then you are able to notice the discrepancy. It reminds me of what happens with super-resolution algorithms for tiny faces.

Doesn't seem to work on black faces, or faces with beards.

It’s pretty amazing they will ask for money but they don’t say what the limitations of their training set are

In fact they have a separate colorisation feature for B&W pics. It turns people white (with a pink hue), and there isn't a setting to let it know that the subject isn't white.

Off topic but if you missed it: Dall-e looks like a great advance in NLU: https://openai.com/blog/dall-e/

My intuition is that like taxiderming a beloved pet, these will feel "not quite right" because of the lack of subtle characteristics that make the person who they are in your memory.

People may use this differently than me, but the examples shown seem to lean towards people that no one living remembers.

Before doing this, think of the consequences of nostalgia.

The past can be like a black hole if you get too close it pulls you again and again until you disappear.

“Get out of here! Go back to Rome. You're young and the world is yours. I'm old. I don't want to hear you talk anymore. I want to hear others talking about you. Don't come back. Don't think about us. Don't look back. Don't write. Don't give in to nostalgia. Forget us all. If you do and you come back, don't come see me. I won't let you in my house. Understand?“ Cinema Paradiso

next step is having them talk, like this deep fake where parents brought back their dead son:

https://www.youtube.com/watch?v=m6I_wEetSck&t=30s

Deep nostalgia audio from the early twentieth century would all sound tinny like it came from a Victrola.

A bit OT: But are any big studious (yet) applying some deep methodology to movie dubbing?

Uncanny valley 2.0:

1. Take the output of thispersondoesnotexist.com

2. Animate the image of the "person" using this tool

3. Profit?

Works surprisingly well!

https://myhr.tg/10yVgL9d

...that everso slightly jittery mouth... spooky

The sign-up that requires Mother's maiden name is a bit creepy

It might be nice for someone to combine one of these with a recoloring model.

I remember initially being surprised when my parents told me that "the past was in color; just the photos were in black and white". Apparently, color film photography didn't really become economical until the 1960s. Being able to see black and white photos of the past with added colors can help give a better sense of what the past actually looked like.

(The usual caveats about the risks of incorrect recoloring apply, of course.)

The same site (MyHeritage) has a colorizer for old photos. An open source version is available as DeOldify https://github.com/jantic/DeOldify

Oh, nice! I was thinking that this looked familiar; looks like DeOldify was submitted back in Nov 2018 (and the author left comments on HN!).

https://news.ycombinator.com/item?id=18363870

There are lots of AI colored, upscaled movies of the 20s, 30s on YouTube. Quite amazing stuff

Technologically, it's a cool project.

While we're on the subject of nostalgia.. did you know nostalgia used to be considered a serious mental illness? Although, it seems, it can do you some good, or so the research says

https://www.nytimes.com/2013/07/09/science/what-is-nostalgia...

Coming soon: generating your favorite movie starring you.

I am feeling scared more than amazed tbh.

Now take a recorded voice snippet of the same subject and have them say hello to you or happy birthday, etc.

Black Mirror comes to life: https://en.wikipedia.org/wiki/Be_Right_Back

> Martha Powell (Hayley Atwell) and Ash Starmer (Domhnall Gleeson) are a young couple who have moved to Ash's remote family house in the countryside. The day after moving in, Ash is killed while returning the hire van. At the funeral, Martha's friend Sarah (Sinead Matthews) talks about a new online service which helped her in a similar situation. Martha yells at her, but Sarah signs Martha up anyway. After discovering she is pregnant, Martha reluctantly tries it out. Using all of Ash's past online communications and social media profiles, the service creates a new virtual "Ash". Starting out with instant messaging, Martha uploads more videos and photos and begins to talk with the artificial Ash over the phone. Martha takes it on countryside walks, talking to it constantly while neglecting her sister's messages and calls.

came here to find this. This sort of animation is beyond the uncanny valley; far enough to wrench someone's heart who has lost someone see their dear one moving again. But alas, too removed from reality.

If you are trying to make me cry, I think you are succeeding :(.

Pretty magical, my parents were amazed.

I just did this with a picture of my mothers mother that died one year after giving birth to her. My mum broke out in tears (of joy). To have a tiny idea of how she looked like.

Yikes. It’s a little scary to think I may have uploaded a photo of myself to Facebook without first reading the TOS and then consulting a futurologist to make sense of its implications.

Imagine they use your appearance to sell insurance to your relatives...

We're not far from Tom Cruise A.I. dinner party

Charlie Sheen would be way more fun, and already loses Turing tests against GPT3

It's not realy deep nostalgia... As in, it could just as easily be anyone's face right.

Why does the sign in require a gender. Not sure what genitalia have to do with using a website.

just tried it with 3 or 4 pictures: some parts of the animation are convincing, but another good part is just too distorted, the tool fails to keep the shape of the head, the person becomes someone else. Good fun tho.

Norman Bates:

https://www.myheritage.com/deep-nostalgia/result/861467271-5...

Reply on news.ycombinator.com