Memers are making deepfakes, and things are getting weird | Hacker News Reader

Memers are making deepfakes, and things are getting weird

technologyreview.com

68 pointsjpindar33 comments

Deep Fakes are merely the next iteration of Photoshop. Before the 90's, faking photographs required extremely labor intensive processes performed by experts. Multiple exposure tricks, splicing, painting.

One could have expected that the development of Photoshop would have been hailed as the end of trustworthy information, but instead we got memes and photo manipulations such as https://www.snopes.com/fact-check/shark-attacks-navy-diver/

I think Deep Fakes are good. They're going to breathe new life into media and make it significantly easier to make films, music, and all kinds of art.

The public will come to understand AI-generated media as a part of daily life. People are not that stupid. The media (the only ones decrying this) need to stop with the Luddite hysteria.

I've actually been quite involved with deep fakes and wrote a few pretty big sites to help democratize the tech (the average person can't run Google colabs and Python) :

https://vo.codes (over 40 celebrity and cartoon voices)

https://trumped.com (just Donald Trump)

I've already got a real time voice conversion software nearly finished. I can do Discord/Team Speak and talk as Trump and Gilbert Gottfried.

I'll probably do the same for video soon.

While what you say is reasonable given a sufficiently long span of time, I think it's also reasonable to expect that there will be very significant growing pains between now and when the public ultimately comes to understand AI-generated media as a part of daily life.

I think we're maybe a year away from that.

Most of these results are being produced from open source models that are difficult to set up and run.

I joined a few Discord servers that were started this year for deep fake video and voice, and they're now huge. We're also just now seeing subreddits explode with content [1]. My website, vo.codes, is starting to take off on Twitter. My phone is blowing up with retweets, mentions, and followers. Youtube is overflowing with the stuff [2]

Give it a year and these results will be everywhere. The kids love it. Memes are going to normalize it.

[1] https://www.reddit.com/r/mediasynthesis

[2] https://www.youtube.com/watch?v=hRlo8_13Zyw

I think you and I are there now. But there's a looooong gap of knowledge between you and me about this stuff versus, say, grandma and grandpa who live out on the farm.

To be clear, I'm not a luddite, I think this tech is/can be extremely fruitful for creatives -- a new paradigm of creativity. But the fruit can also be plucked by many propagandists, and IMO there can be serious consequences right now unless we educate people about this stuff right now. I'd argue that the approach can't be fear alone, but it also can't be all sunshine and rainbows.

Edit: I also just want to say that the work you're doing/referencing is very helpful toward this education

> Gilbert Gottfried

How much would you charge to not make it easier to release more of that nails-on-chalkboard voice into the world?

I know HN isn't the place for flippant jokes, but I would literally pay for Gilbert Gottfried narrating serious things like Planet Earth. I think it would be amazing.

https://vo.codes has you covered. You can also use David Attenborough's voice to narrate Gilbert Gottfried standup.

You say that "People are not that stupid" but have you looked at all the obvious fake/false information being passed around on facebook. I think that shows that people really are that stupid.

I think it shows that nothing about Photoshop or deepfakes matters in the slighest since the kinds of content that people are falling for now is a stock photo with some text under it.

Seems to me that that the end of your comment supports the parent commenter's point, rather than showing the irrelevance of deepfakes for the soundness of public opinion — that people are so easily and constantly deluded now gives little reason to hope for a discerning public in the face of stronger fakery

Much of that misinformation is also made with far cruder tools than a well-trained GAN. People will be fooled by what they want to be fooled by. Remember how Republicans thought Nancy Pelosi showed up drunk to some political event because someone just slowed down the video? Or the whole wave of cleverly-edited "expose" videos on ACORN and Planned Parenthood done by the Project Veritas guys? The reality is, anyone who believed those videos already had a negative opinion of those organizations and people that jived with the lie they believed.

It doesn't matter if Nancy Pelosi wasn't actually drunk, she's a lunatic anyway!

It doesn't matter if Planned Parenthood wasn't actually selling body parts for profit, they're baby killers anyway!

It doesn't matter if ACORN wasn't actually telling pimps how to file taxes, they're Obama lovers anyway!

The problem is not that it's possible to convincingly falsify video evidence. That's the kind of problem you need to deal with as the judge of a courtroom, and courts already know how to deal with manipulated and falsified evidence in other contexts: strong monetary and criminal penalties. However, we're talking about the people at large, who are not unimpassioned triers of fact but people who have strong opinions that they are angry about. The court of public opinion is not an impartial or fair court, it is a cacophony of angry people shouting into the void until society gives them a perceived win.

Look at QAnon nonsense: 4chan convinced a good chunk of conservative America that Trump is fighting a global child cannibalism ring by just writing a bunch of incomprehensibly long-winded stories. People aren't stupid, they're angry.

It's funny you mention the court system as knowing how to deal with manipulated and falsified evidence, but you imply the videos from Project Veritas, &c., were "cleverly-edited exposes," when it is the courts - and even Planned Parenthood, while in a legal case - that have found those videos to be factual, and to not have any particular omissions, and to be valid evidence:

https://www.texasattorneygeneral.gov/sites/default/files/ima...

page 10:

> The district court stated, inaccurately, that the CMP video had not been authenticated and suggested that it may have been edited.

> In fact, the record reflects that OIG had submitted a report from a forensic firm concluding that the video was authentic and not deceptively edited. And the plaintiffs did not identify any particular omission or addition in the video footage.

[Planned Parenthood are the plaintiffs in this specific case.]

A bit more information on this subject: https://www.youtube.com/watch?v=POdbu4bp5rQ

Including some of the videos referenced themselves: https://www.youtube.com/c/TheCenterforMedicalProgress/videos

Re your QAnon comment at the end -- I'm not sure if you saw, but it seems that (as of today) people are defending Q on HN as well :( Is disinformation against HN policies @dang, and if not, shouldn't it be?

> People are not that stupid.

We act as if there has to be some majority of opinion for anything to be impacted.

> Bruening miscalculated the mood of the nation after six months of economic depression. The Nazis won 18.3 percent of the vote and became the second largest political party in the country.[1]

> Roughly one-third of Americans who have heard about it see truth in the conspiracy theory that the COVID-19 outbreak was intentionally planned by people in power [2]

We're still fighting a conspiracy from the Middle Ages.[3] And it has overt political support again.[4]

[1] https://encyclopedia.ushmm.org/content/en/article/the-nazi-r...

[2] https://www.journalism.org/2020/06/29/three-months-in-many-a...

[3] https://www.wired.com/story/opinion-the-dark-virality-of-a-h...

[4] https://www.mediamatters.org/qanon-conspiracy-theory/qanon-m...

I think this analogy is too simplistic. Video fundamentally has a level of engagement and connection to human senses and perception of reality that goes far beyond photography. By the time that fast/convenient photo manipulation tools became widely available (i.e. not Photoshop 1990-2010, but well into the 2010s), video had already long replaced photo as the average human's gold standard for evidence.

Even just thinking about the past 5 years (or even just the past 5 months) think of how much impact citizen phone video has had just in the U.S. – outside of a courtroom. Even well before George Floyd, the power and utility of bystander footage has arguably been the strongest factor in public advocacy for state and local police to mandate officer-worn body cams.

If you are grounding your thoughts in the expectation that "people are not that stupid" you are setting yourself up for failure.

> merely the next iteration of Photoshop

I think you're significantly understating the impact here.

You're talking mostly about content-creation, and in this context there are a lot of positives; I certainly agree with your statement:

> They're going to breathe new life into media and make it significantly easier to make films, music, and all kinds of art.

BUT. I don't think you can, with a straight face, say that there are going to be no negative impacts of this technology. You make a claim that this is just like Photoshopping in the 90s, but I think the key difference here is that in the '90s, there was a fallback medium; when photos became unreliable, you still had video, which couldn't easily be faked.

Looking forward to the next 5-10 years, how does anyone figure out what's true when photos, video, and voice can all be trivially faked by anyone? What reliably medium can we fall back to? This could lead to a real epistemological crisis!

> The public will come to understand AI-generated media as a part of daily life. People are not that stupid.

My problem with this position is that it hand-waves away some pretty fundamental changes to the fabric of our reality. How exactly do you propose that people are going to understand the world, post-'fakes? If any video you see in your twitter feed could be fake, how can anyone make any inferences about the world outside of their own immediate direct experience?

Video of the same event from multiple angles will probably be safe for quite some time (timing and lighting mismatches

Video from a single source already has the issue that the person showing you the video can start and end the video at whatever time they want to.

There was a time before video, before photographs, before writing even.

So, not new at all. We've got thousands of years experience in dealing with truth under limited circumstances with false claims. Consensus reality was a passing fad.

I'm not trying to be adversarial, but you must truly live in a bubble if you think that the average person will be able to catch on to all the ways this can and will be abused. There are already huge problems with people being influenced by very obvious botnet trolling. It has serious political implications. You're probably just not around or in the social media sphere of the kind of people who are into e.g. flat earth conspiracies.

I am curious about how this will affect advertising and e-commerce. There was a post about bots generating cosplays and clothing in real time for purchase. That seems more interesting usage of this tech.

Motivate people to buy something by showing them personalized video ads of someone they idolize calling for purchase?

Facebook has all the data and network it needs to pull deepfake advertisement and shopping business.

I think this is actually a good thing. The more the average person is familiar with deepfake tech, the less likely they are to get misled.

I like the philosophy, but tricking people isn't very hard. Apparently, some shaking camera footage of a computer monitor was all it took to discredit a political dissident https://www.nytimes.com/2020/08/23/world/asia/cambodia-faceb...

It seems like as soon as it confirms their bias they are willing to ignore if something is real or not.

You say "they" like this isn't something that every single person in the world does constantly -- nobody is immune to it. Confirmation bias is just a derogatory term for people having priors.

Yeah, and it really is everyone. For example, remember when the White House Press Secretary tweeted a video that had apparently been doctored to make it look like a journalist had attacked one of their staffers? There was a really clear, convincing comparison that overlaid a semi-transparent version of that video on the original, showing that it leapt ahead in the part where his hand went near her because it had been sped up to make the interaction look more violent. Many people spread it, including YouTube debunker Captain Disillusion with this message: "Getting lots of requests on the topic, but there's nothing for me to examine. This person has examined it, very simply and clearly. And I concur." https://twitter.com/cdisillusion/status/1060564297103917056

Now, having compared the two videos myself and not found anything like this, obviously I had a look at it - and worked out the trick almost immediately. The comparison had the overlay ahead of the original by the same number of frames the whole way through from frame 1, but it was so faint that during normal viewing the difference was only visible in fast-moving parts, creating the illusion it leapt ahead in those parts. Easiest trick imaginable, could probably have been done trivially with film a century ago, and fooled someone who'd built a career and reputation around debunking faked videos along with many others.

(As for the White House video, that's a long story but the short version is that it seemed to be pretty much exactly what it purported to be, and the only actual "doctoring" was probably the obvious, intentional stuff like repeating parts of the video zoomed in.)

This assumes you even need to go that far to successfully mislead people. The most authoritative sources of information in the world right now today are the written and spoken word. I mean people wouldn't just lie right?

For the world you're describing to make sense people would actually have to demand video evidence to believe something.

That still leaves the other problem - people will eventually stop believing real video evidence. Cops will claim videos of their abuse are deepfakes, and people will believe them. Politicians will claim that recordings of them soliciting bribes are deepfakes, and they'll be off the hook. Concrete evidence of misdeeds is one of our best ways to hold powerful people accountable, but with deepfakes they can simply flood the web with fake incriminating evidence and we'll become so accustomed to ignoring it that we ignore real evidence as well.

They already had that to get people to ignore video evidence - just scream "It is out of context!" while committing acts that could never be excused under any circumstances.

This doesn't look very convincing.

Also, I didn't know you could run python in google drive.

>There’s a telltale wonkiness to the faces in the videos made with this algorithm, which makes its handiwork easy to recognize; that is part of the deepfakes’ humor. These imperfections—and the surrealist quality

Part of most meme's style is purposeful bad photo editing, so the poor deepfake quality is part of the format.

Not Drive, Colab. Colab is a VM service. (Although you can mount your Google Drive, and many Colab users will.)

Relevant shameless art project plug. Nlp+voice clone model of Joe Biden. Streaming on hold until interactive chat is tested and complete.

Disclaimer: I'm not a fan of either of the two parties. Trump version to come.

https://twitch.tv/biden_unleashed

Reply on news.ycombinator.com