Stopping deepfake news with an AI algorithm that can tell when a face doesnt fit | Hacker News Reader

Stopping deepfake news with an AI algorithm that can tell when a face doesnt fit

spie.org

146 pointsrustoo100 comments

The problem will increasingly be "whose algorithm do we believe". Internet has revealed that people believe mostly what they want to. We have seen that a large subset of people are believing Bill Gates is behind the pandemic. Why would masses somehow be more rational in picking the most rigorous and objective neural network to recognize deepfakes, than they are in making sense of the world in general? In the end we will have multiple competing entities claiming to have the best deepfake recognition, all with their own agenda

I think religion might have proved that people will believe whatever they want to first. The book Sapiens really opened my eyes to people.

Well it took me 35 years but I escaped religion despite being indoctrinated from a young age. And leaving was a difficult choice but education and evidence were the keys that freed me.

Piercing cognitive biases is hard, maybe harder if those most in need feel forced.

Be kind to yourself and know that a fair swathe of the early years of life are necessarily of low agency.

I could have clarified more- I’m saying that even with these sort of deep fakes that breaking people out might be just as hard. Especially if objective proof becomes “malleable” similar to how religion can lack objective proof, manipulating people in a new way means we need methods.

Your point seemed clear enough to me. Perhaps my response drifted too far. I see education as a common solution to the "it's all relative" and "no one can know for sure" shortcuts some people take to hold on to what's more comfortable to believe.

And just as religion may often teach people to turn off their critical thinking skills they can be taught to use them again. And human brains evolved to recognize unhealthy or unnatural facial imagery for millennium. So unmasking the more successful techniques may be enough to break us out of the "anything can be fake" malaise.

> Piercing cognitive biases is hard, maybe harder if those most in need feel forced.

I think about this article (post?) called “How to Change a Mind”[1] a lot. It’s about a woman whose husband was in a religious cult and how he finally realized it.

[1]https://forge.medium.com/how-to-change-a-mind-1774681b9369

From some of the terminology used, I'm pretty sure the cult in question are the Jehovah's Witnesses.

Thankfully that one seems to be dying off. Their membership is in decline, the internet has made it more and more likely that any potential convert has already heard the horror stories.

This is on point, but we need to be clear on the direction of causality, which I believe you have correct, though I'll try to state it more clearly:

The human mind and body is predisposed for social cohesion through shared narrative, and this exposed a weakness in our psychology: we are, as groups, extraordinarily gullible.

As a result we have religion, conspiracy theories, politics, and crime.

It remains to be seen whether this is an evolutionarily fit strategy.

This is a very low-resolution understanding of what a religion is. It has much more depth to it. (The writer of Sapiens actively tries to ignore this by reducing every single belief humans have (including religions) to random meme propagation.)

That religions are just mind-parasites has become a meme on its own. To me, the book "Darwin's Cathedral" made a very good case that we should look at religions as adaptations that allowed creating larger societies. And so I finally came to peace with religion. Will stay atheist, thank you for asking.

Another very thorough analysis I can recommend is Jordan Peterson's youtube series called "Psychological significance of biblical stories". There are about 10 2-3 hour lectures, but they are packed with great content and original thoughts. People think JP is some political activist because of his media portrayal, but in his core he is a student of a much more substantial content that has little to do with politics.

>> people will believe whatever they want to first.

Including Harari. There's more wrong in that book than there is right, aside from basic facts that people should already know.

Chriatopher Ryan (Civilized to Death & Sex at Dawn) explores the few compelling points made by Sapiens in depth and without all of the disingenuity and naive philosphy present in Sapiens.

Because YouTube, Facebook will flag these videos before they become too widespread.

You're missing a sarcasm mark.

But seriously: at best it's going to be a big "maybe, it depends". These services bounce between PR disasters of "complicit purveyors of propaganda" and "oppressive algorithms destroying free speech", so you can't rely on them to be on either side.

As deepfakes get better, the algorithms will have worse detection rate, so it'll end with Zuck testifying to congress that Facebook doesn't want to decide what is real, so everyone can post deepfakes all day long.

> You're missing a sarcasm mark.

I don’t see any sarcasm there.

Perhaps the wild zigzag is a hint that the people casting accusations are utterly full of shit and just want them to bend the knee to their unpleasable whims?

A video being taken down just corroborates their belief that there is some deep conspiracy behind X event

"I trust Google and FB, and I would like them to decide what is true or not for me instead of deciding myself"

Amazes me how people are okay with giving these massive corporations so much power and control over our lives - voluntarily. Most would be rightfully horrified if Walmart or Citibank was to decide what you get to watch or not tomorrow. But somehow, if Alphabet or Apple does it, it's all okay.

The tech world in general needs to wake up about the overreaching power and influence tech companies have on our lives.

I can only imagine it's because what seems obvious to us is really not noticeable to 99% of people.

You have to remember we are in a little bubble of our own for better or worse, most of HN readers principles are generally aligned against this kind of centralisation and we highlight and amplify anything we see that matches; I'm guessing your average FB user is more focused on the value proposition of these services they are consuming and wont give the corporations behind them much of a thought until it hits big news.

Much of it depends on the way people interact with businesses. Big Banks are bad because people can literally see overdraft fees and all sorts of surcharges being tacked onto their accounts, often in shady ways. Big Oil is bad because they literally pollute the environment and routinely spill millions of gallons of oil in the ocean.

In contrast, for the average person, Google is just the free little search engine that shows you all the answers and also gives you the free phone software. Whatever egregious privacy violations they do are wrapped up in a layer of abstraction that, if you don't get technology, can be hard to grasp.

The next generation is smarter though. They understand technology fundamentally and know that there's no such thing as "free".

That dumb meme about "big tech controlling your mind" ironically seems to be doing a far better job of what it accuses others. It has made people /unlearn/ how the internet and private property work.

Even with more egregious examples of moral wrong and stupidity like shutting down child molestation survivor support groups or just bad decisions like buying up Tumblr and trying to sanitize it weren't like but accepted as fundamentally how it works when it is their servers. There is no viable alternative to stop them from setting their rules. Even if they die there is nothing stopping them from taking their ball and going home.

Aren't digital signatures the best solution we have for this problem?

Commercial entities want to have the priviledge of being able to modify the content submitted by content creators, but the culture of trusting i.e. Twitter over client verification needs to change.

If the technology is too hard to understand for the masses it's unlikely to provide much resistance to desinformation campaigns.

The technology isn't hard to understand, but even if it were so what?

How many people who watch a streaming video know... anything at all about how that content is delivered, at all, at any layer.

The problem is, as OP noted, the existing gate keeps profit from being able to manipulate us, so convincing them to deliver signed content is extraordinarily unlikely.

Digital signatures would only match of the contents match bit for bit. This means that you can't recompress/remux/resize the video. This will probably provide a very poor ux for mobile or other bandwidth limited users. Also, if the video was used as part of another video (eg. TV broadcast), you'd either have to splice the digitally signed video into your existing stream (not modifying the bits at all), or provide the digitally signed original as an appendix. In either case, you'll need tooling to make verification easy, which doesn't exist today.

Fingerprinting can take care of the recompression problems.

Remuxing is a hard problem, as it can easily take people out of context, at the same time it's very important to summarize the videos, as people have limited time.

Right now what I see though is that Facebook/Google/Twitter aren't even trying to do the bare minimum end-to-end authentication that Whatsapp/Telegram/Signal already does (create a private key on the end devices, and sign the content to verify the authenticity of the publisher).

Requiring HTTPS was a great first step for tech companies to protect people and from ISPs. But they do nothing to protect people from themselves being compromised (the Twitter incident was a great proof for this).

> Fingerprinting can take care of the recompression problems.

That only opens another problem: adversarial attacks that look totally different, but has a similar fingerprint

>Right now what I see though is that Facebook/Google/Twitter aren't even trying to do the bare minimum end-to-end authentication that Whatsapp/Telegram/Signal already does (create a private key on the end devices, and sign the content to verify the authenticity of the publisher).

>Requiring HTTPS was a great first step for tech companies to protect people and from ISPs. But they do nothing to protect people from themselves being compromised (the Twitter incident was a great proof for this).

Mainly because to 99.9% of users, there's no difference between a message that's signed by the author, and a link to a tweet that's made by the author. Even if you're the 0.1% that do care about public key cryptography it doesn't matter because you're trusting the site to do the verification and key management. It's not like PGP where you can get the public keys and verify yourself.

There's also the problem that people simply don't care. Have you seen how many photoshopped tweets end up on social media? If people are willing believe a screenshot without a link (which is trivially easy to add and verify), what makes you think they won't believe a screenshot without a signature?

Why can't you have digital signatures of the original and have an ML model that can evaluate whether or not it matches? You could run a cloud service to host the signed originals and continuously update the model.

There's some cryptographic work in this direction. PhotoProof allows a photographer to prove that the image they're presenting is, e.g., the cropped version of the true original image they took. Video is still way out there, but at least people are thinking about this.

https://www.cs.tau.ac.il/~tromer/papers/photoproof-oakland16...

Signature chain for the transformations. The video host signs the pre-recompressed video and provides the original signature.

Signing it doesn't mean it wasn't altered at some point and does nothing to actually verify the content - only if a source is who they say they are. And the key distribution is another issue.

Digital signatures are so far away from a solution for this problem they might as well be in a different universe.

One of the main ways people share content is by screenshotting it on their phone, or literally taking a photo of a computer screen with their mobile. Good luck using a digital signature to solve that.

This is absolutely right. I've become frustrated with people trying to fight misinformation by just telling anonymous strangers "facts". Algorithms are not going to help win the argument. There are a lot of people who hate those who they perceive to be "elite technocrats". And its not completely unwarranted!

Its a really tricky challenge. The "steady state" of conversations online is mutual distrust. Unless we handle this underlying issue, clever algorithms like this will be ignored.

I'm reminded of the Jonathan Swift quote:

>Reasoning will never make a man correct an ill opinion, which by reasoning he never acquired

The problem that specifically deepfakes and generally fake news highlights is that the general public does not have the aptitude, knowledge, time, or motivation to be a true arbiter of facts. We have traditionally outsourced that role to journalists. As a society we have lost faith in journalistic institutions so the onus is now back on the individual. We need a way to offload that truth-finding responsibility back onto a third party that can be trusted. Any system which requires more aptitude, knowledge, time, or motivation from the general public is unlikely to work because lacking those is exactly what got us into this mess.

The key part of your sentence being "a third party that can be trusted."

A trustworthy third party is NOT the group that is "the most smart". People trust locals that they relate to. If we are going to offload responsibility, I personally think it needs to be in the old fashioned democratic pluralism style that we had half a century ago. Local leaders know the people in the area better, and they have the time and motivation to hear the interests of more people.

> trying to fight misinformation by just telling anonymous strangers "facts"

It’s even worse when you realize that most mistruths are factual. You can very easily lie with facts.

The texas sharpshooter falaft is a great example.

You get 500 samples, take 10 good samples, and say “Look! An objective truth in 10 samples we have success!”. You omit the other 490

You never said a lie, facts only, but the interpretation you lead people towards is a lie.

Or a more common example: Thing Y increases your risk of X by up to ten times!!!

Risk goes from 0.0001% to 0.001%. It’s completely irrelevant and you got a nice scary clickbait with objective facts.

For me, one reason not to freak out about this is Photoshop.

It's been around for decades. Perfect photographic fakes (literally called "photoshops") are possible. Yet is there a great crisis of fake photographs taking over the news? Not really. The actual "fake news" barely even bother with photoshop (and those for whom it works, don't care about the quality). It's somehow still fairly easy to get context and at some point, you just have to trust a news outlet, just like you had to trust them for text-based news.

All we see is a trend of making it easier for people to subscribe to a bubble of "news" that fits their world view. The quality of the fakeness barely factors into this.

The other reason not to freak out about this is that we're already far down the path the people think deep fakes are creating.

Reddit and Twitter and the internet prove that people will react to an image of text by the tens of thousands, going straight into their brain. All of us have been guilty of this at some point.

To me it's naive to freak out about Photoshop and video deepfakes because it reveals that you're completely unaware of the degree of "shallowfaking". A screenshot of a headline or tweet spreads in a way that a deepfaked video can't, and it apparently goes right past our bullshit detector in a way that video can't.

The issue with deepfakes is this: at least in the other cases you outline, it's possible to be skeptical and weigh evidence-- including direct photographic and video evidence that is unlikely to be safe (while it may have been possible to fake before, it's been labor intensive and presents more risk that it'd have been messed up). In turn, we can evaluate the credibility of media sources based on how they agree with other evidence that we have directly evaluated.

As it becomes possible to fake more and more stuff, we're going to lose that. Different silos are going to have different, meticulously supported versions of the facts with an equivalent degree of direct-evidential and reputational support. And it's not clear to me as an individual one ever disentangles this.

While video is currently unlikely to be entirely fabricated at this point, you might be surprised at what can be accomplished with camera angles, choosing the start and end times of your clip, and cuts and reordering of clips: for a silly example of this, see "Obama sings Call Me Maybe"[1]. Videos can also be staged with varying levels of convincingness.

If you are digging for evidence, the source of your information must be taken into account. Deepfakes will probably be detectable in retrospect, even if they are not detectable at the time of publication, so if you stick with sources with a long history of only publishing real information, you're probably still fine even in the age of deepfakes.

The arguable lack of journalistic sources with a demonstrated history of integrity is its own problem, but not one that is made significantly worse by the presence of deepfakes.

[1] https://m.youtube.com/watch?v=hX1YVzdnpEc

This would be more concerning if the public was actually well informed currently. At present, most media companies have wildly lowered their content standards, normalized clickbait titles that sometimes outright lie, all the while ramping up "it's an op-ed so it's not a lie it's just an opinion!" pieces. At the same time we've never been more interested in what government agencies have to say including what they have to say about the government itself. This world of a semi-informed populace doesn't exist, the public is already choosing information from sources that appeal to their emotions rather than logic. What person on the street actually has a nuanced understanding of an industry they're not participating in? How many have any real idea what government contractor work actually looks like in those same industries or understands what the warzones their government operates in are actually like? Most don't even understand other cultures that vaguely resemble their own, just look at how few westerners have the vaguest understanding of sex culture across asia. The only thing people know anymore is memes and rhetorical factoids. You can't poison a toxic well.

Maybe, but I've found that a lot of viral images are doctored in one way or another.

Here are some links to recent examples of doctored photos. I know I've seen at least the deceptive image of cops "pointing a gun at children" when they were actually not on the front page of Reddit, so it's not like manipulated images have no effect:

https://www.hackerfactor.com/blog/index.php?/archives/884-Pr...

http://hackerfactor.com/blog/index.php?/archives/891-Count-o...

That said, remember that not all alterations are digital:

https://www.hackerfactor.com/blog/index.php?/archives/590-Un...

That blog is frightening and informative - it gives some idea of the sheer scale of misinformation we have to cope with now.

> is there a great crisis of fake photographs taking over the news?

Well, yes. Many newsworthy photos and video clips are doctored, and they sway the public discourse in subtle ways. That we haven't yet seen significant public outcry is as much a testament to their subtlety as it is to their effectiveness.

I'd argue voice and video are substantially more persuasive than static images

This is by design.

Reflexive Control Theory: https://www.rand.org/blog/2020/07/how-you-can-fight-russias-...

Now we can use this new AI to train another AI that can defeat it. And so continues the great cat vs mouse chase.

I don't think this is a problem that is ever going to be solved. Deep fakes will become more and more popular, and harder to detect.

I feel like it’s time to bring in Heinlein’s Fair Witness concept: https://www.urbandictionary.com/define.php?term=Fair%20Witne...

If you think about it, when we had relatively few journalists and their reputation was on the line with every story they wrote they in a way were fair witnesses. Since we have moved to a hybrid model of professional journalists and a huge caste of YouTubers, podcasters, bloggers, and twitters, it has become impossible to hold everyone to a high journalistic standard. At the same time Fox News has brought on an era of politicized news so the bias is now inherent. Maybe if we had a few hundred fair witnesses that offered a professional service we could have a way to verify facts. Then again, it’s one thing to have a fair witness present facts, it’s another to have that witness’s likeness be deep faked and broadcast all over the web saying something they didn’t actually say.

Another solution may be regulatory: anyone broadcasting a deep fake with a huge disclaimer that it is a fake gets a 20% of their net worth fine. Draconian but I can see it working.

We need something, but Fair Witnesses seem both exploitable and unachievable given the way human memories work.

“Expert witnesses” then?

“Expert” is a misused term, much like “exponential”.

I see the value of the press as being as much of an intelligence agency as the combined CIA, NSA, and FBI, but serving the electorate and investigating all those with power — politicians, police, military, religious, business, and so on — so that the electorate can make informed decisions.

I don’t know how to get there from here. Heck, I don’t even know if we’ve ever really been there, or if the press has always been comfortably manipulated by those with power.

But yeah, I assume genuine experts are part of the solution.

Off topic, but 'exponential decrease' is my favourite misuse, simply because exponential decrease is slower than a linear (steady) decrease.

Even if achievable it seem inhumane. And besides it seems far more achievable to just have trustworthy journalists, even though that's already difficult in itself.

Antibiotics resistance is also a cat and mouse game. It's still useful to keep playing the game though.

The difference is that all the bacteria can become resistant to the antibiotic soon after it is created.

As can deepfakes. All it takes is an additional step to optimise wrt the model that tries to catch it.

Yeah, I was dismissing the analogy with antibiotics because usually it takes quite some time between creating the antibiotic and germs being resistant. But with deepfakes the arms race is almost instantaneous. The moment something appears to tell apart deepfakes, the moment people can train deepfakes against that something.

Perhaps the deepfake checking should be a third-party service? So you can only check so many deepfakes per day, limiting these attacks (i.e., you can't realistically put the checking inside a training loop). Just an idea ...

Ref:

http://www.gutenberg.org/ebooks/29579

Robert Sheckley's Watchbird

Looking at the diagram, this appears to be just an LSTM slapped on top of a CNN. If so, I'm failing to see any novelty in this approach. RNNs on top of CNNs have been used before, including for deepfake detection. See for example: https://arxiv.org/abs/1905.00582

The recent DeepFake Detection Challenge threw a lot of manpower at the problem earlier this year, BTW.

Isn't this the idea of getting better results using the adversarial network approach? So this would be inherently ill-suited to stopping deepfake news, yeah?

it's great at making them better, yes

If the people trying to stop deepfakes can do it so can the people trying to produce them anyway. The best we can do is some way of digitally signing videos and software displays it as authentic or not but then they just make fake youtube style sites that say its authentic. The problem will go on and on.... Fake news will always spread for one reason or the next.

> but then they just make fake youtube style sites that say its authentic.

Even simpler, someone will build an optical setup that sends deepfake pixels onto a signing image sensor.

Deepfakes are generated by a generative adversarial network.

There are two networks: a generator, and a discriminator.

The generator generates a result, the discriminator evaluates that result.

This AI that detect fakes faces could be used to train the discriminator so that the GAN generates even better results.

AI novice here. It would seem to me that the detection algorithm can be repurposed into making the original less detectable. A recursion that never ends. An advantage quickly becomes disadvantage.

Worse. All information - even facts and truth - has been subverted. What happens when there is no trust? Is this not a road to the New Darker Ages?

That's what I'm thinking. I don't want to diminish the value of this research, but this cat-and-mouse game is like a GAN[0] with extra steps.

[0]: https://en.wikipedia.org/wiki/Generative_adversarial_network

There is a simpler alternative imo: consider any news that do not list all of their sources as fake news. Treat them as if they were scientific papers.

We should also look towards simpler alternatives. ( For example some days back I also read about a profile made on a social media platform with a "generated" face )

1) For image platforms to prevent "generated" profile with fake faces, may be request 2 profile pictures and not one. A task that is difficult to achieve at high resolution.

2) For deep fake videos .. May be the idea be to fight the "video" part and not the "deep fake" part. By that I mean "signed" content. ABC news should "sign" generated videos and so should other publishing houses. So that any other source claiming forged voice or certain kind of "faked" content are not able to do so.

Can god create a fake so deep that they themselves can’t detect it?

DNA will set them apart

Cool tech, but most people will have tuned out what we know as "the news" (or become jaded to its purpose beyond entertainment) before a tool like this is necessary.

This will just be like anti virus. Peole will start to train their algos to fool the current algo that is is being used. We will be a never ending fight.

That's GAN training in a nutshell.

Are deepfake face swaps a real problem yet? I can’t recall any major controversies in recent history that were caused by a deepfake face swap.

> This technique can be used to create compromising videos of virtually anyone, including celebrities, politicians, and corporate public figures.

I’ve read a lot of concerned comments like this but I haven’t read about any real world examples of a controversy caused by a deepfake.

Hey, we all know that this will not be the end of deep fakes. It will just another channel of information to think about. Now we have to care about whether this algorithm is correct, or maybe it is also manipulated by the other political party to claim that the others' picture is fake...

But it had to be created at some point, it had to exist. This is just an inevitable next step of the progress.

The challenge with algos like this is that they could be used to claim events that actually did happen didn't. So as an example credible/convincing footage of Jeffrey Epstein by a pool in Paraguay last week could be 'identified as a deep fake' and discredited, despite other supporting facts and information that lent credence to veracity.

People believe what they want to believe.

Just a basic search and 5 minutes in wikipedia about a topic will make you better than most people who sprout nonsence.

But you got to have a right mindset.

Human brain hates change. You have to force the bitch to take in new information and process it and prepare it for further info on the topic.

Maybe advanced DRMs will be a way forward ? Have camera fingerprint / sign the video, and then every editing videos fingerprint & sign the changes performed, and send everything to a ledger ? Only way to make sure a video is actually coming from the real world...

We don’t need this for photographs. Why do we need it for video?

Seems like an AI arms race just begun

It's just a slobbering infant trying to roll off the bed at this point.

The battleground of what is real and fake will very quickly move into 'superhuman' realms of sensitivity, leaving us meaty minions as spectators trying to figure out which AI to trust.

I used the AI to destroy the AI

Someone just found the other half of their deep fake GAN

What if it gives false positive results? What if it tags non-deepfake news as deepfake-news ?

Now we need to stop believing our eyes twice: first when we see a natural-looking picture but know it is a deepfake, and second when we see a natural-looking picture we believe is not a deepfake, but the computer tells us it is.

This seems like it’s “adversarial” to deep fakes. I wonder what that can be used for.

This seems like test driven development, but applied on a much larger scale.

If this is differentiable (which it seems like it should be), it could be used adversarially to create better deepfakes, BTW.

see also "Training a deep learning model for deepfake detection" ( https://news.ycombinator.com/item?id=22433711 )

Can be used as an algorithm for a new discriminator in the GAN.

as Zizek would say, "And then you use this as the basis for a GAN that optimises for faces that do fit, making better deepfakes and so on and so on"

This is getting stupid. People.. aging, cancer, Alzheimer's still exist. Maybe time to reprioritize & organize efforts?

Those are completely different deep specalties and best funded in parallel anyway given diminishing returns. Complaining about something that irrelevant to capabilities is like saying brain surgeons suck at inventing green energy.

this is the virus killer war all over again

I find all this concern a little bit overblown. Yes, it is a problem that we can make extremely convincing fakes, but we've had fakes that can fool non-experts for a very long time. The Soviet Union doctored a great many photographs in a way invisible to me (examples: https://en.wikipedia.org/wiki/Censorship_of_images_in_the_So...). Why are we more concerned about this fakery than about airbrushing?

The solution is the same as it's always been: stick to trustworthy sources and insist that all evidence is traced and corroborated. It remains easy to learn the truth as long as you make a good faith effort to do so. In the worst case we can just stop trusting photographs altogether. We got by just fine before the camera was invented; we can do fine after it becomes obsolete.

Because good fake video is now possible for the first time, good video was nearly impossible to fake. Because people believe certain leaders who are willing to use false evidence. Because the populace is not prepared to be that discerning.

The deepfakes which support the officially blessed narrative will not receive scrutiny, while authentic videos will be "proved" to be deep fakes by some black box machination.

But we've known not to trust anything we didn't personally see ourselves for many decades.

Reply on news.ycombinator.com