What evidence is there that deepfakes are harmful and not just distasteful?
Generating offense and disgust is not necessary inherently harmful.
Generating offense and disgust is not necessary inherently harmful.
First of all, and most relevant to the discussion, that website handles single pictures. They don't do videos, which means I can pedantically classify the result as a "photoshop" instead of a "deepfake".
As for the results themselves: I uploaded a couple pictures of me. I can't see the full result because I am not a paying user, but even in the blurry preview I can see that it's a fake. One of them even fused my clothes and my skins, which I guess could be a turn-on for those who are into Lovecraftian horrors.
I won't deny that it's technically interesting, but I wouldn't start a moral panic over its results.
Whatever happened to mystery, and desire, and letting your imagination run wild trying to fill in the blanks?
It's like the computer is being used to narrow our horizons, not broaden them.
All the consternation about deepfakes ignores this higher-order effect.
I am old enough to remember the times when "photoshopped" became a verb, and yet I don't remember a "Photoshop epidemic" where people blackmailed other people with production-quality fake pictures. I mean sure, it happened, but not at a major scale.
If the past is any indication we will indeed have deepfakes of personal connections, but their quality will rank from "bad" to "okay", and that will be it.
I remember at the time this was a pretty common thing, in high schools, so perhaps you just managed to avoid it.
Photos went through the same thing: they used to labor-intensive to fake and generally accepted at face value, now that everyone can easily manipulate them with photoshop people are much more suspicious. That's how we got to people using video as proof instead of simple images. Now faces in videos will lose this trust, and people will find more trustworthy signals.
A low-trust society is a significantly less productive and safe place to be. We don't want to go back there.
Agreed.
> people will find more trustworthy signals.
I would really like to believe this but the world is clearly experiencing a massive crisis of trust because we have failed to find reliable trustworthy signals now that broadcast Internet communication makes it so easy to create and spread nonsense.
Hmm.
I think you could tell a couple of stories:
- "It used to be that media was used to broadcast journalism to the public, but now the internet allows lies to be spread faster than they can be caught".
- "It used to be that what the public found out was gatekeeped by a particular set of people, but now the internet allows the public unfiltered access to the truth".
The public used to have a limited set of narratives they had access to because creating and broadcasting them was expensive. That naturally limited the set of narratives available to those who had some level of power and wealth. The upside, if you feel that any level of power or wealth is ever earned, is that those broadcasting have presumably done at least something of value to society to partially earn the right to broadcast. The downside is that much of it may be unearned, or the value they provided may have only been to the elite few at the expense of the many.
Today, narratives are so cheap to produce and broadcast that the public can choose whichever one suits their whim or preconceived notions.
It's not entirely clear to me which of these scenarios is better. They both clearly have deep structural flaws.
Throughout human history we have used effort and cost of communicating in a given medium as an indicator of the legitimacy being contained therein. At various points the bar to entry for disseminating information using said mediums has dropped precipitously causing relative chaos as we acclimated to the new reality. Often times institutions or groups that had power and were using the difficulty of disseminating information to maintain that power were left with less when the dust settled.
Right now we're in the middle of one of those precipitous drops. From realistic special effects and green screens to celebrity porn modern computer tech is enabling high production value content with incredibly minimal expenditure compared to what would have been required in the past. That's gonna shake things up. Maybe it'll be a nothing-burger like photography. Maybe we'll have a century of religious wars like the printing press.
Society will be fine in the long term.
Alternatively, once the quality becomes good enough, "it was a deepfake" will be an entirely reasonable defense. Not just in actual courts (which is what I fear we will be heading towards, that even horrible crimes such as raping children will go unpunished or underpunished in countries where possession even of drawn/animated material is a crime), but also in the "court of public opinion".
> The video, which shows a rendering of the Ukrainian president appearing to tell his soldiers to lay down their arms and surrender the fight against Russia, is a so-called deepfake
https://www.npr.org/2022/03/16/1087062648/deepfake-video-zel...
There are many earlier (questionably "deep fake") examples: https://www.washingtonpost.com/technology/2020/08/03/nancy-p...
Did the Ukranian soldiers do this following the deepfake video? As far as I understand it, most of them are still fighting the Russians. What damage or harm occurred here?
I am pretty sure modern military command and control does not rely very heavily on "the video looked like it was the president saying it".
We know from the economics of scams that any social engineering attack that is cheap enough to do and can be made plausibly urgent enough to get people to not question it can be hugely profitable. Deepfake blackmail has the potential to check off both of these boxes:
1. Porn adds plausible urgency by defaming the victim. People will really, really do anything to avoid being associated with the porn business, as can be seen by how many copyright trolls specifically troll with porn as opposed to other copyrighted works.
2. It is getting cheaper to train deepfake models and public social media photography provides a large repository of images that could be used to generate face swaps.
Right now I haven't heard of this kind of blackmail actually happening yet. This is primarily because deepfakes still require manual labor to cut out large numbers of faces, and the training process takes a while.
However, these are both surmountable problems.
Ransomware has been a technically possible form of malware for some time, but the problem was getting money back to the thief in a way that was difficult to trace. The kinds of scams that target technically illiterate users use money mules, but that relies on the unwillingness of the criminal justice system to prosecute the kinds of crimes that hurt the weakest in society. Bitcoin made ransomware a lot less risky of a crime, to the point where a lot of criminals targeted large businesses that would ordinarily be protected by the law.
The only thing keeping it from being profitable to deepfake all of Facebook into a bunch of porn is that it would be expensive to do so. However, that's purely a technical problem, and the thing we know about technology is that it almost always gets cheaper over time.
[0] Tom Scott was actually going to do a speculative fiction video about this exact hypothetical... before the whole deepfake porn thing took off and spooked him out of it.
How about doing something about
* blackmailers * people giving in to blackmail * people trusting random videos on the internet
rather than trying to put the genie back into the bottle?
- Woman can't go topless in most places.
- Porn has age limits
- In some locations profanity is restricted in public
And these are rules from the government that clearly violate the first admendment but are accepted because of the level of offense and disgust they provoke.
Meanwhile the number of ethical uses is relatively quite small. Maybe there is a novel use of this tech that will actually be useful, but I can't see how being able to easily impersonate someone else will be good. Maybe someone here has some ideas.
Thank you for clarifying that. Even in the entertainment world (resurrecting dead actors) it's still a fake (a.k.a. fraud) just that we don't depend on it in a way with real consequences. Deepfakes are inherently fraud/deception.
Further down the thread there is an example of a political deepfake used to spread disinformation. I don't really care about distaste, it's authenticity (the truth) that is under attack by these. On the upside, they can offer a bit of entertainment.
It's not even a problem of regulation since the bad guys will always have deepfakes. The cat is out of the bag already.