Creating a deepfake took two weeks and cost $552
arstechnica.com
arstechnica.com
15 years ago that would've been science fiction, today it's a free gimmicky feature in an app running on commodity hardware. There are also apps that will smooth your skin or change the shape of your eyes in real time.
All these technologies are evolving at a ridiculously fast pace, I'm sure that a decade from now you'll be able to do a much more convincing fake in a fraction of the time. We're getting deeper and deeper into a post-truth world.
Also, I'm fascinated by the idea of using AI to detect the use of AI. How good will your deep fake have to be in order to fool a deep fake detection AI?
"deepfake news"
I can just hear people saying it now. Joking aside, and not taking in ANY ethical/unethical misuse considerations this software is capable of, it is really cool tech.
Sure, and that's part of the danger. Convincing deep fakes will be used to delegitimize mainstream news organizations, as with what happened with Dan Rather - a trusted source presented faked documents.
And we've been hit by tabloids, spam and so on before. None of that is new. Overwhelmed by content is/was/will be a problem that is solved very simply: people limit the distribution lanes they consume, and naturally establish "trust networks" and hierarchies.
Nyt didn't become "trusted" by accident. People aren't as dumb as you seem to think.
Anyways the problem is of distribution, not that a lie can be told. If we can't trust our distribution lanes to actually reflect the institutions we're trusting, then we can't establish our trust networks.
And this is a real problem: our distribution lanes are divorced now from our actual information distrubuters; medium, Facebook, Twitter, etc fuck around with our trust networks, randomly injecting their own bullshit into our feeds and messing around with feed-order based on non-trust metrics (eg money paid) such that they've become fairly unreliable.
And our classical trustable institutions have become less trustworthy, as they flounder about trying to make sense of the "digital age", and have so far done so in a pathetic fashion
When that network breaks down, you only need a single good lie to do the damage.
If 2004 news could be fooled by a fake that took a couple of minutes to make, I’m not that worried that today’s news is any better.
The people who care about the 'legitimacy' of a news organization are moderates who are passively engaged and are being rapidly driven to the margins. The 'objective truth' of the matter is less relevant than how it is received. Activist politics is en vogue thanks to the feelings of helplessness brought on by globalization, and if you're not an activist your views are not really an important part of the process anymore. You'll have to pick one of the two options that the activists come up with because they won't show up to vote for a centrist candidate. In either case, a moderate platform cannot win.
For the record, I feel the above situation is incredibly dystopian; but we're hardly the first country where pathos supersedes logos and ethos.
The only difference is today, they don't retire out of shame, they just keep on doing it, keeping viewership via emotional draw rather than rational news. Real news has been dead since that incident in my view. It's mostly state propaganda, regardless of party.
This is, itself, "fake news". I can't find any indication that other mainstream media outlets defend the documents "to this day". The reporting largely seems to be "certainly smells like bullshit, and we can point to specific issues with them that seem to support that".
https://www.nytimes.com/2004/09/13/opinion/those-discredited... cites a wide variety of news organizations looking into them and finding problems, stating "When the mainstream press checked the sources mentioned or ignored by '60 Minutes II,' the story came apart." and noting Newsweek specifically calling them "discredited".
And what about when this happens at a state-sponsored level (This is absolutely already in development, if it hasn’t happened already...). “Did you see that Fox News exclusive video leaked from the Bernie Sanders campaign? Yeah he was holding a private dinner for a bunch of Billionaires. They say they got it from some Russian oligarch he invited. So much for grass roots change, I think I’ll just stay home this Election Day.” Or the corollary: “That smoking-gun tape the Do-Nothing Dems have of me at Putin’s personal sex trafficking center, where the two of us are laughing about having Epstein take the hit? Fake news! They’re called deepfakes, and they make these just absolutely incredible fake videos. Not real at all though. Only trust what the Government-sanctioned media says!”
As per your other point, given the GANs that are used to make these are precisely a deep fake AI fooling a deep fake detection AI, aren’t we already there? Further, when the government is making the generator, a privately trained discriminator has no chance. How would you even train that? Get the government to give you a bunch of generated fake media so that you can develop a really spot-on model? I don’t think so.
Two predictions:
Deepfakes will play a roll in the 2020 election.
Deepfakes will play a roll in every election thereafter.
If this doesn’t concern you, I don’t know what would. The thing with the “post-truth world” is that once people think we’re in it, we are.
Long have documents and news been the subject of fraud. And for a few decades photo alterations have been common. Now we are adding video to the list.
But there is nothing new about the faking of libelous material (eg Killian documents 2004)
How many people do you think actually read those “Killian documents”? How many people do you think a state-sponsored social media campaign built on the most accessible media would reach?
"With software there are only two outcomes: either the users control the program or the program controls the users. If the program is not under user control, and the developer controls everything, then the program is an instrument of unjust power, applied by the powers that be."
Deepfakes are just a continuation of "fake news" (formerly known as propaganda), and the end outcome is we either succumb to those who use it, or adapt - filtering out the bs isn't easy, but it's not an insurmountable task, either.
It's won't upend the world, but we are losing a hugely helpful tool that has been relied upon for 60 years.
The technology to fake paper ballots is trivial, but they still (mostly) work fine for elections, because we've developed strict rules for how they need to be handled in order to retain credibility.
But recently I read "Tlön, Uqbar, Orbis Tertius", and I thought that the point Borges was making could be easily described as critique of "fake-news post-realism of today", except it was 80 years ago. And while the notion of this problem not being new at first seemed weird to me, eventually I figured this is pretty natural: while 80 (or 800 for that matter) years ago there wasn't any such tools to manipulate information on that scale, nobody had the access to information on that scale in the first place. So anyone could be fooled by more powerful actor anyway. Offensive tools of any time are roughly the same as the defensive tools.
It's just that these 20 years at the turn of the century were somewhat the sweet spot, when we had this "Internet for people like us", so it was us who was fooling the less knowledgeable people, while keeping the belief that we are relatively safe from being maleficently disinformed by somebody else. And now we can clearly see it's not necessarily the case anymore, ao it feels like something new.
But it really isn't. As always.
The best generator and the best detector are actually part of the same model! If you create a better detector, you are making the generator better at the same time so you have not accomplished anything.
"GANs are a clever way of training a generative model by framing the problem as a supervised learning problem with two sub-models: the generator model that we train to generate new examples, and the discriminator model that tries to classify examples as either real (from the domain) or fake (generated). The two models are trained together in a zero-sum game, adversarial, until the discriminator model is fooled about half the time, meaning the generator model is generating plausible examples."
https://machinelearningmastery.com/what-are-generative-adver...
And it does seem that the human-visible weaknesses of many currently published deepfake examples are those that would be hard for the current discriminator architectures to detect, not those which would be harder to generate than any other part of the fake.
The film "Mr. Smith Goes to Washington" came out in 1939. It should be mandatory viewing.
1) The researched human behavior where "1000 lies must be equal to 1 truth".
https://www.researchgate.net/publication/256486791_Credibili...
2) Monopolizing the information stream: the attention grabbing effectiveness of deepfake videos to convey fake messages and its novelty incentive to share them on social media, added to the fear-of-missing-out they generate on viewers and the fact that "the effort required to refute b* is 10 times greater than the effort required to produce it":
https://www.telegraphindia.com/india/social-media-fatigue-an...
3) voter apathy caused by news fatigue due to the sheer volume of fake news, to which deepfake adds a new vector:
https://www.nytimes.com/2019/11/18/us/polls-media-fake-news....,
https://www.datadriveninvestor.com/2019/07/04/chronicle-fati...
4) A majority of youngsters can't really spot (or care) about fake anymore:
https://qz.com/1750839/most-young-2020-us-election-voters-ca...
[1]: https://www.theverge.com/2019/10/22/20927521/snap-growth-ear...
None of that actually happened because people learned to be skeptical, and to care about who took the photo, the chain of custody, and the controls to prevent digital alteration.
Maybe we need something similar but more rigorous for video. The camera itself could encode a PKI signature in every frame of the original. If subsequent edits preserve frames as is, then the viewer could determine how many frames were cut or duplicated.
If subsequent edits crop, recolor or otherwise alter the frames, that editor could encode its own digital signature so the viewer could judge how much to trust their edits.
Is there evidence that deepfake videos will be worse than photos? My (naive) sense is that even for an AI, editing a photo will always be easier than editing a video with sound.
If Photoshop and deep-fake images aren't a problem, shouldn't we be less concerned about video?
This may be true in courts of law, but it absolutely is not when it comes to general public consumption of pictures.
The 9/11 Tourist Guy picture that duped a lot of people 18 years ago looks like a ridiculous MS paint edit today. https://en.wikipedia.org/wiki/Tourist_guy
Likewise, a deepfake is worth a thousand lies.
Photo editing has been used pretty much since the beginning of mainstream photography. You should never "trust" pictures for anything serious if you don't have absolute trust in the source.
Besides it's one thing to edit a still image and a whole other to forge a long video, complete with audio. Look at those movies these past few years using 3D models to insert younger versions of actors (Terminator and Star Wars come to mind). They must have invested millions into it, they have very talented artists and it's pretty good but it's still quite easy to spot the fakeness of it.
The Daily Mail was at it from around WW1, perhaps earlier. Though their main WW1 fake was, of course, the cadaver factories. Surprisingly the Times bought that tale too.
And why doesn't the government keep track of the cameras and security keys so they can trace back any images they do not approve of?
Are we though? Or is it just becoming much more obvious the limitations of relying on the media in general. This very well might lead to a better understanding of truth, and our personal responsibility in understanding it.
At the very least, it's not a foregone conclusion that this will lead to worse outcomes than what we have today. It might be what tips us in a new and better direction.
What we'll continue to see are rising sects of flat-earthers, anti-vaxxers, cultists, polarization and deniying of anything not-in-my-bubble, a complete lack of trust in journalism and media, and futile attempts of fact-checking while we're all drowning in fake news shared endlessy on unsuspecting users. It takes a lot of effort to show something was manipulated digitally, and you'll have to fact-check by yourself, that doesn't scale at all.
The fact that we're more aware of flat-earthers today, is probably a good thing too. I'm not 100% sure that there are more of them today than before, or if they're just more noticeable. But in either case, we should be very careful to draw too many conclusions when we look around today because we're living in a time of great transition. We're basically in our teenage years of this technology.
Just as you shouldn't judge an adult by who he was at 14, I don't think we should assume we know what the adult internet will look like because we see some acne on it today.
Even this is too simple. Many people have lived under propaganda regimes, yes, many have bought into them, and many people have seen through them.
Each new wave of information technology provides new ways to “pull the wool” over peoples’ eyes.
And though a propaganda machine has power... like all things it creates and equal an opposite power that eventually destroys it. The question for individuals is only: where am I in the cycle, how long will this cycle last, and what is my role to play?
Anywhere you have trust, there's a potential for abuse. A seemingly natural response to abuse of trust is to find a lapse in the victim's vigilance and start strapping on your own armor. I listened to a podcast this fall (sorry, I don't recall for certain--my best guess is that it was peter pomerantsev on Ezra Klein's podcast) where the guest suggests that some institutions/relations (like markets) depend on trust to produce value and may be undermined or degraded if enough participants take an individualist caveat emptor approach.
It is hard to know, in a complex adaptive system, what new equilibria will emerge downstream from a big disruption. We certainly could emerge through the other side with better epistemologies. But I don't see it as a given. Thinking is hard. Proving anything is harder. Skepticism can be good (and I suspect we need more--though I don't think it's terribly well distributed or applied at the moment...), but it's also easy; it requires no trust, knowledge, thinking, rigor, logic, or proof.
So, I'm troubled by the possibility we'll dig ourselves into a deep world-shrinking skepticism (a callous) of anything we didn't/can't personally experience. It can be well-founded on real abuses of trust. It may even be the sum of individual rational choices.
Neither would necessarily keep it from unmooring things we mistook for solid ground. And, when someone finally notices one drifting, there may not be enough trust left to agree on fundamentals like: is it adrift? is that a problem? was it ever tied down? is there anything we can do? should we wait for a bit to see if it drifts closer first? should we act?
But i'm terrified of the current loud support toward censorship and authoritarian suppression of incorrect ideas. We have a lot of historical evidence of just how bloody that always turns out. So I choose to believe in the power of openness and competition of ideas.
As others have posted above, we will never have a time where it's possible or desirable for each of us to validate the truth of every claim, so that means we will have to find trustworthy arbiters of truth. I think that this should happen by people showing they are trustworthy and earning respect, rather than using authoritarian control to shut down those with differing ideas.
Much of the media today has lost such respect, and deservedly so. You can not fix this by suppression of free-speech, but only by creating honest organizations that aren't afraid to respectfully show the ideas of those who see things differently -- even if it's just refute them. Organizations that _trust_ the viewer to decide between fairly articulated positions.
The contempt for our fellow man when we say that bad ideas must never see the light of day or enjoy the same platform as government-approved ideas is a deeply paternalistic and ultimately anti democratic idea. It's elitism at its worst.
(For example, we need to be able to trust that they have a sincere opinion and aren't paid shills for a corporate lobby on a PR mission to waste our time, emotional energy, column-inches, and attention.)
If you feel a growing sense of dread as it retreats, it's a good time to consider whether you know others who have expressed a similar sense of dread that other things in our world have come unmoored, and whether they could use some help keeping them tied down.
Even society is shown to be untrustworthy, and the social processes of science.
Thus some reactionaries want to replace trust with authority. Or markets, which is another name for social proof, but with much less transparency.
Ultimately the other parts of trust are verification and reputation, which are both not unassailable. (Sybil style attacks, bribery, institutional and social biases.)
But, our saving grace might be simpler: cooperation without trust is possible and might be as efficient if things get bad enough.
The thing is, people do not check information that matches their viewpoint for validity. Because being wrong feels bad.
Maybe I’m being paranoid but it seems that the public is being conditioned to believe deepfakes are so good prior to a certain cache of video recordings being made public
So essentially you’re training a network how to compress an image down to a very tiny representation, and then uncompress it as accurately to the original as possible.
You train two of these: one for the original face, and one for the target face. Then, you compress with the “source” autoencoder, and then uncompress with the “target” autoencoder. And, voila, ‘source‘ face becomes ‘target’ face.
For example, the canonical way to train deep machine translation systems is to make use of large parallel corpora - large set of translation pairs. In the case where this data may not exist, for structurally similar languages, one approach is to learn unsupervised word embeddings independently (i.e. using word2vec) which only require monolingual data and to find a transformation that aligns the two spaces. In many cases, neural networks still struggle with this task. However, some work in the last few years has demonstrated that a few seed word-translations is all you need to apply simple algorithms that can learn a linear or simple nonlinear transformation that work pretty well!
[1] Here is a method that proposes locally linear transformation maps for alignment: http://nakashole.com/papers/2018-emnlp-norm.pdf
One of my favorite related papers: https://arxiv.org/pdf/1703.00848.pdf
In this case, the body is being entirely ignored by the autoencoder. The way it works is by first doing a machine vision pass to isolate just faces and process on that region for both sets of source imagery.
Uh.. That's what a DeepFake is? Look up the definition.
> Deepfakes (a portmanteau of "deep learning" and "fake"[1]) are media that take a person in an existing image or video and replace them with someone else's likeness using artificial neural networks.
"Likeness" refers to appearance in general, not just the face.
Still, what's remarkable is that a neophyte like me can create fairly convincing video so quickly and for so little money. And there's every reason to think deepfake technology will continue to get better, faster, and cheaper in the coming years."
> In the long run, the larger risk may be that public attitudes swing too far in the opposite direction: that the possibility of deepfakes completely destroys public trust in video evidence. Certain politicians are already in the habit of dismissing media criticism as "fake news." That tactic will only become more effective as public awareness of deepfake technology grows.
Deepfakes themselves will not disrupt democracies or cause the much anticipated chaos. Deepfakes are in fact pointless for agitprop / disinfo campaigns:
"cheapfakes beat deepfakes" --@thegrugq
https://twitter.com/thegrugq/status/1206404680223358976
https://medium.com/@thegrugq/cheap-fakes-beat-deep-fakes-b1a...
What is fascinating (and uncommon) is, that the damage isn't delivered with any specific deepfake itself. But the mere awareness that the technology exists, and that they could _potentially_ be used, is enough to change our risk perception. Even if the ratio cheapfakes/deepfakes usage in actual disinfo campaigns remains tiny percentage, deepfakes make much better headlines. (in a similar way the infosec industry is obsessed with 0days - but most of the risks are coming from people clicking on links in email)
Maybe one day we will look at deepfakes like "art". If pictures and videos can be art, and if "all art is propaganda" (see Orwell), then deepfakes too should be considered art, no?
I understand the point of the article is “look how much I did with so little.” And it’s really good in terms of how much was done.
But the final deliverable isn’t at all convincing. Clickbait title.
ex - Bill Hader/Tom Cruise - https://www.youtube.com/watch?v=VWrhRBb-1Ig
I think one thing that could have helped the reporter is if they had reencoded the original Zuckerberg footage into something with a lower bitrate. The fine details/imperfections on his forehead contrasted with the lower fidelity of data's face provides a little too much contrast. It doesn't help that most of the source data for Data's face is lower quality, plus he has so much makeup pancaked on his face. I think if those factors were accounted for, the reporter's results would have been much more convincing even though he's a deepfake neophyte.
I'm NOT claiming that this is the case, I'm just saying we don't have a reason to believe why this can't be the case.
There really are two separate questions though:
- Can experts with a big budget create videos that even equally expert analysis can't tell from the real thing? (Maybe yes, maybe no. But it's hard.
- With modest skill and budget, are people increasingly able to create fake videos that stand up a cursory glance and can fool people as a result? (Absolutely--or at least we're getting close.)
But that already exists, DeepFake isn't a fundamental development for that. What is the "big thing" about deepfake is that we can imitate speeches of people in leadership positions. For this, we can get really really close; but it's unclear if it'll ever be realistic enough to be a problem in court cases, politics etc...
https://www.youtube.com/watch?v=Wm3squcz7Aw
Voice actor reads a poem in voices of famous people; his face gets swapped with the actor's face; the combination of voice and face makes it very believable.
The absolute autofellatio state of hype media these days. Ha-ha, look at how this autistic CEO looks like a robot, and shame on you for imagining what Charlize Theron looks like naked. Tune in next week where I try and fail to launch an ERC-token, and use that experience to argue that cryptocurrency adoption will be delayed.
I guess I always assumed the 'deep' in deepfake meant it was convincing, but upon further consideration it's probably just a nod to DNNs. Well played sir.
Even better (though I'm not sure this technology exists yet), some some kind of "rolling signature", available running along side the video in metadata, that a media player can use to validate the authenticity of the video at that point in time. Then you don't need the full video to be able to verify the source.
Remember, technology to fake voices has already be done. https://www.youtube.com/watch?v=I3l4XLZ59iw
Certainly interesting times ahead.
But for now deepfakes seem like a 2018 meme that died out.