Creating sexually explicit deepfake images to be made offence in UK
theguardian.com
theguardian.com
This is an interesting one because it violates the vague principle of "whatever done by consenting adults in the privacy of their own home is cool".
Like what if someone pastes my face ona naked body, or imagines me naked, or has a dream about me flying through the window naked and writes it in their private dream diary.
I think it's getting into hairy legal territory if the "infringing material" is only seen by the otherwise innocent creator on an airgapped computer.
Instead of looking for those answers in the text of Farris’ summary of the law, we should look at the text of the law itself.
Edit: https://bills.parliament.uk/bills/3511/publications probably has the text we're looking for, somewhere.
A private deepfake image is not the same.
There are plenty of incredibly disturbing videos and images of some of the most gruesome crimes committed and they are lawfully accessible and obtainable on the open Internet, including some such images on Wikipedia. The documenting of a crime is not and has never been illegal.
On a related note, I assume there is already a lot of deepfake CSAM in certain circles. If that is ruled legal, it will make finding and prosecuting real life CSAM creators much more difficult.
At least in the US.
Well if those images are illegal, you're right that there's no way to produce them legally, but that's because they are on their face illegal.
If you mean there is no way to produce CSAM that doesn't break other laws, I think you're wrong - e.g. see the case of teenagers taking nude pictures of themselves. That is an otherwise legal thing to do, but having those images is illegal, as I understand it, and you can search Google for many examples of teens prosecuted for sexting images of themselves. I don't know if anyone has been prosecuted for merely having nude images of themselves, but that might just be because no one would even know about it, necessarily.
https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...
I wonder if deepfake technology is going to change our stance on that. With enough motivation Congress could take it out of the hands of SCOTUS.
https://en.m.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coal...
Maybe it could be read that way by some, but that's not why the term was adopted and it's not why the term is advocated for by anti-sexual violence organizations.
https://inhope.org/EN/articles/child-sexual-abuse-material
https://www.rainn.org/news/what-child-sexual-abuse-material-...
https://www.thorn.org/blog/decoding-the-language-of-child-se...
As for the fact that making it simulated renders the term meaningless, the entire point of a simulation is that it is not the actual thing that is happening but rather an imitative representation of something that could be real. Presumably the abuse towards a child could be real, and hence it's possible to imitate that abuse in various forms of media. It is that imitation that constitutes the simulation.
If you argue that it makes the term meaningless, then you must accept that the simulation of anything is meaningless.
I think that what is being argued. Put 'simulated' in front of any legal term and it becomes meaningless for its legal usage. 'Simulated larceny' is meaningless. 'Simulated murder' is meaningless.
There are plenty of laws that make it an offense to perform a simulation. Nitpicking the term simulation as being meaningless, whether in a legal context or otherwise, isn't a particularly noteworthy discussion to have. I think most people understand the meaning of simulated CSAM and can have an opinion on whether it should be legal or illegal without being pedantic about the word "simulated".
You are really stretching here, and it is not 'pedantic' to ask that specific terms have specific meanings especially when it is a legal issue.
You are laughably misinformed on this matter in a way that is quite incomprehensible. You are welcome to refer to the legal definition as per the actual legal text instead of continuing to make rather bizarre claims on a topic where it's clear you know nothing about:
https://www.congress.gov/bill/108th-congress/senate-bill/151...
Even without having to refer to the actual text, I am confident most reasonable minded people understand the meaning of the term "simulated" and can take a position on whether it should be legal or not without arguing whether simulation means training materials or whether the term is legally meaningless.
The fact that you don't is all I need to know that continuing this conversation with you is entirely pointless.
I think the issue here is the between consenting adults part, deepfakes in this instance are probably not between consenting adults. I also think it's important to think about the reality of how this likely plays out. If someone deepfakes me and faps and I never find out and they never tell anyone, it would be quite hard for there to be action. So if someone is deepfaking you and you know about it, there must be a logical path there, and none of them look good for the deepfaker. Can't say I know where I land on the issue, and I've thought about it for days and hours.
People are acting like a "deepfake of a person" is something different than a drawing, painting, song, poem or any other artistic creation inspired by, based on, or with elements drawn from a real person.
"If someone is deepfaking you" is such a weird imaginary problem. Someone might be imagining you. Someone might see you in a coffee shop while writing a screenplay and use the outfit you're wearing to describe a character in their story. Who cares?
If someone is committing fraud by trying to pass off a deepfake as a real photo, then we already have laws around fraud, defamation, harassment to cover that. If someone makes a deepfake that looks similar to a real person for their own amusement, that's no different than writing fan-fiction or sketching in a notebook.
Distribution of an image is not an effect of its creation.
Deepfakes in this instance are not /between/ anyone. Only one person even knows of it.
I feel like "without consent" to be pretty key here.
Which is basically what we're seeing here.
IF the artwork is porn.
In fact, if you're not old enough, it's just flat out illegal. You'll go to prison for making that artwork. And you arguing that it's "art" is not going to find much resonance in US courts at least. You're likely to fail the Miller test if US citizens are the judges.
For example, cooking meth.
I do think the other examples you gave have one significant and fundamental difference - no one would mistake those things as being real. The problem with deepfakes is these have a similar social impact and capcity for harm as real nudes do, only there is no element of consent. You could ruin someones life quite easily with deep fakes, but not with your lewd poetry or sketches. So it seems reasonable to me to ban these with a blanket ban.
I would prefer we lived in a world where that wasnt neccesary, but for every technology you must imagine the most sinister and exploitative application and realise that someone will try that and worse. How do you even begin to combat deepfakes and the social problems they bring? how many bodies are considered acceptable until we get it right? Possibly, its best to over reach a little then relax the grip rather than to take it slow, given what is on the line for some.
A ban won't stop bad people from making them anyway. On the other hand, proliferation of deepfakes would make the existence of certain real or fake photos insignificant, as everyone would accept that anyone could make them easily.
sigh.
I file this whole issue under: "This is why we can't have nice things".
People who create models like that ruin everything for everyone.
Courts have previously deemed cartoon imagery of children to be equivalent to a real child, so considering the deepfake to be a real person that needs to give consent is not without precedent.
The implied answer here appears to be “no” and the reason this is deemed necessary is an emergent effect of scale of being able to do this at such volume and quality.
Saying you can’t draw porn of people without regard to implied authenticity or medium seems somehow more reasonable to me.
Although I might call out the opposite problem of making it a crime to request a sfw picture of a specific person and have the model generate something porny
Personally I think head-swap porns should be handled with defamation logic than general anti-porn laws. I find it more likely that I would be disgusted by deepfake porn of mine if I were to become such public figure that warrants it, than appreciating it, but it feels weirder that local police would be deciding whether am I to like it or not.
We don't even have a tool that could reliably ID the subject of such a processed visual depiction.
The judge.
https://en.wikipedia.org/wiki/I_know_it_when_I_see_it
The justice system is a computer made out of humans executing written programs called laws. As we've seen from decades of computer vision research, writing out an image classifier in a finite set of symbols is intractable. So instead the justice system queries an oracle (a human) to do the image classification.
Maybe in the future there will be a way to compile a learned image classifier into a set of directions in English and we will have laws that are followed by pages of what is essentially a small artificial neural network represented as an English flowchart, but I doubt it, that sounds dumber than asking a person to do it. You just have to make sure you're picking the right person to do the deciding.
[1] https://www.gov.uk/government/news/government-cracks-down-on...
Also UK: Criminalizes 'thoughtcrime'.
The fact that a representation of a thing itself is illegal is quite backwards and ought to be at least seriously debated more seriously than in the moral panic we have now. Otherwise, we’re back to punishing people for their creations, whether that would be ideas, writings, paintings, or digital files, just because some people do not like it.
But creating an image that can be watched in the real world, is more than "thought".
What exactly is "on you" in this scenario? In your world is someone with Neuralink allowed to take their brain images and use that as an outline for a sketch they're drawing or you're not allowed to think of anyone naked ever again?
Also UK: make above real enough that it has a term
that's some admirable British consistency. Unironically. Honestly.
When the world is prosperous, people tend to vote for more liberal policies. And when economy is struggling, people tend to vote more conservatively. So we see more authoritarian policies like these.
On one hand it seems liberal, wanting to see a change that gives individuals who might be subject of deepfakes more autonomy.
On the other hand, AI is changing the world and it seems like an effort to try and hold the things around AI in place as to not seem them change.
It could go either way. So could the state of the economy right now, for that matter.
I don’t think this framing is very useful. Modern conservatives don’t have problems with change, they just want that change to go their way. They are perfectly able to innovate and push new doctrines or legal theories when it suits them. And on the other hand they do not care about preserving anything besides an idealised vision of “traditional values”, which are not as old as they seem.
Looking at the actual policies and arguments, the spectrum is between oligarchy (or dictatorship for the most extremes) and democracy. It makes much more sense that way.
> On one hand it seems liberal, wanting to see a change that gives individuals who might be subject of deepfakes more autonomy.
It’s also a significant limitation of personal freedom in the case of a victimless crime, which is anything but liberal.
In fairness, some political parties who subscribe to reactionaryism operate under a capital C Conservative banner to try and muddy the waters, but to consider them conservative because of that is like considering the Democratic People's Republic of Korea democratic.
Indeed, that’s exactly what they are. But they prefer calling themselves conservatives. It’s better from a marketing perspective.
To be fair, there are not many politicians that could be called old-fashioned conservatives in the US. Most of them are a subset of the republican establishment and they were mostly wiped out in the last couple of years.
> to consider them conservative because of that is like considering the Democratic People's Republic of Korea democratic.
Definitely! Unfortunately, that’s how the semantics went, though. In the same way as what most people call “liberals” now are very different from who liberals used to be (they used to be all about capitalism and free enterprise, for example).
“A decent job” is subject to interpretation, but I find very difficult to argue that the Magna Carta was the first, even if it was very significant.
Also, what you say is very true. The Magna Carta was a hastily cobbled document to protect the power balances and keep the serfs in their place.
That's japan.
I feel bad for people who are harmed by the creation and publication of such content, but it's not obvious to me that it's possible to combat it effectively using the legal system. Then again, maybe such laws don't have to be completely effective, so maybe it's sufficient that sharing such content cannot be done publicly in order to deter the most obvious abuses.
Do you think in the very long term we'll remain so attached to protecting our likeness or will continued digitization of reality bring forth a loosening of such cultural norms?
People can do lots of things easily and are prevented by the legal system. I don't agree with the part of the law that makes this a crime if it isn't shared (who is being hurt in that instance?) but using an argument that essentially says 'it is hard to prevent people from doing something so don't even bother criminalizing it' is nonsense.
Children younger than 10 need to be taught the concept of consent and privacy, and how upsetting it is for people whose privacy and consent are violated. It ties into cliques and bullying, too. Teens can be brutal in their bullying and fake nude pictures will be added to the arsenal of abusive actions.
Some of this stems from unrestricted usage of computers by minors who don't understand the impact their actions might have. Better parental tools and parents/guardians who take this topic seriously would be beneficial too. (I'm not arguing for some locked-down console instead of standard PCs, I just think that parents need to be able to know what their kids are doing online)
That's not a good argument for creating this legislature. It couldn't really be enforced before (photoshopped images of celebrities), and it certainly can't be enforced now when any "script kiddie" will be able to generate terabytes worth of deepfakes in a week, and launch them to circulate on the internet anonymously.
Unless you institute some draconian measures like only being able to access the internet with a passport and having 10% of your workforce work in censorship.
No privacy is violated by this act.
Consent violation is nonsensical.
The word "just" here is doing a lot of work considering scale is usually the difference between something being harmless and dangerous. I think this is a blind spot with technical types as to the intent of the law. We ban things all the time that are easy to do, like shoplifting or graffiti. It's not a failing of those laws that marker pens are cheap and blank walls are everywhere.
Worth noting, I haven't read the proposed law in question and don't have an opinion on it yet. I just don't think the reasoning of something being easy means we shouldn't attempt to legislate it if it has demonstrable harm to society.
Consider libel laws. (Because, if you think about it, "explicit deepfakes" are essentially "visual libel" — scandalous claims made through visual rather than written "evidence.")
For a statement to be considered libellous, it has to be false, known as false to the writer/distributor of the statement, and written/distributed with an intent to cause harm.
But there's an implicit proposition there — that certain statements can cause harm. Which has its own requirements: for a statement to be potentially harmful, it has to make a claim that if believed would result in reputational damage; and it has to be convincing / be likely to be believed by the common man.
A statement isn't libellous if it's prima-facie false — that is, if nobody is likely to believe it.
Photoshopping someone's face onto some random person's body is easy enough to spot, that it has usually been considered (the visual equivalent of) a prima-facie false statement.
A deepfake is only bad insofar as it becomes increasingly impossible to spot the problems with it, and so becomes (the visual equivalent of) a prima-facie true statement — actual libel.
Which country's libel laws. They are not all the same. The UK is famous for libel actions that would not be countenanced by US courts.
This law is about giving victims the means to combat the harm done to them
Creation alone harms exactly nobody.
What is misogynistic is thinking that women's bodies are so shameful that using a computer algorithm to generate visual depictions of them without any interaction with the woman is inherently degrading or inherently harmful.
Yes, they can be used for harassment. But so can a text message "XXX is a sl*t". So can a crude pencil sketch.
The misogyny is in the intention and use, not any sense of realistically depicting the female body.
Surely the basis of a lucrative new extortion model. The most dangerous computer virus will be one that creates an image on your PC and alerts the extortionist.
For those who talk about privacy in your own home, there's plenty of stuff that's done in peoples homes that's illegal and no one cares about. This law's only going to come into effect when these start being shared which I hope we can all agree is a bad thing if done unconsensually.
Not at all. Else the criminalisation would be of those services.
But AI porn? There are so many gray areas and loopholes. This just feels so friggin' half-baked. It just feels like something that should be handled on a case by case basis. If somebody creates AI porn of somebody and uses it for clearly harmful purposes (blackmail, humilating somebody to their coworkers, whatever) then that should be punished.
But, outlawing them altogether? We tolerate pornographic fanfiction and traditional pornographic illustrations of real adults, even though these might be upsetting to the people depicted therein.
Therefore the issue with deepfakes is that not only are these things upsetting, but they might be mistaken for actual photos and videos.
- What if I create deepfake porn of $FAMOUS_ACTRESS but it's sort of crappy, and nobody could reasonably mistake it for an actual photo of $FAMOUS_ACTRESS?
- What if I create deepfake porn of $FAMOUS_ACTRESS and it's extremely photorealistic, but has clearly fantastical elements? (e.g. she is naked on the surface of the moon, she is having sex with Godzilla, etc)
- What if I create deepfake porn of $FAMOUS_ACTRESS and it's realistic, but I've clearly indicated via caption or some other means that it's a synthetic image and not an actual photo of $FAMOUS_ACTRESS?
- What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate out her face?
- What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate the explicit parts?
- How explicit is explicit? What if I've depicted $FAMOUS_ACTRESS in a swimsuit or other revealing outfit that is perfectly legal to wear in public? (reminder: going topless or even nude is legal in a lot of places too)
- What if I've generated AI porn of $FAMOUS_ACTRESS that is based on publicly available images? eg: nude scenes from her movies, etc
- What if I create deepfake porn and you can't tell who it is? Do we go on a search for the victim?
- What if I create AI-generated porn and it accidentally looks like somebody real?
- What if I create deepfake porn of $FAMOUS_ACTRESS but I never say it's her?
- What if I create deepfake porn of $FAMOUS_ACTRESS but I change a detail or two? Perhaps I depict her with the "wrong" eye color, or add a scar, or something. So is this deepfake porn of "her" or more of a derivative work?
---- The harm:
Many people do not seem to understand the harm. Deepfakes are used for sexual pleasure, and used to degrade and dehumanize people resulting in victims getting traumatised. Victims often do not get enough support and the trauma may stay for a lifetime, and some victims may end their lives due to this.
These victims get sexually harassed, e.g. one woman is continuously being harassed after someone generated deepfakes of her and people recognised her from those images https://www.reddit.com/r/legaladvice/comments/1c1pzjo/ai_gen...
In r/singularity thread this is discussed and unfortunately many people think it's okay. https://www.reddit.com/r/singularity/comments/1c5c8pp/uk_cri...
---- Thoughtcrime:
It is worth to discuss the 'thoughtcrime' where the generated content is not shared and where the deepfaked person is not harassed. Quote from the article:
> The creation of a deepfake image will be an offence regardless of whether the creator intended to share it [..]
• Revenge Porn does not make it illegal to store pornographic content as long as there was consent. However, Revenge Porn prohibits sharing no longer consensual content.
• Sharing the generated content is not necessary in order to degrade, dehumanize and harass victims, thus still cause harm. On the other hand, if the deepfaked people are not harmed then it is a bit different, which may seem like a "thoughtcrime" and possibly may be.
• People cannot guarantee the data they store won't get leaked. Both Revenge Porn and Deepfakes have the same issue. However, in Revenge Porn it is easier to see who created the content and enforce the law, while in deepfakes harder.
I get that deepfakes are superimposing a non-consenting face on a (presumably) consenting body, but how does that matter? Is it that "visually lying" is a legal loophole?
That's not how deepfakes work, since nothing is superimposed.
Models like Stable Diffusion build a training set out of several images of a face (e.g. with something like LORA), and reduce it to a set of algorithms. That algorithm is a sophisticated way of expressing "green shirt, black hair, large chin, etc" It's not actually storing photos of faces. It then unmasks a new face out noise using those blueprints. You can change the blueprints, for example, I could ask SD to give me an image with gigantic rabbit ears.
To a layperson it might look the same since the technology is so new, but it's not in any way analogous to a photograph, it's analogous to a painter who saw someone, wrote down some notes on a notepad, went to their studio and painted them. The painting might look realistic but it's not the same as the original photo.
More reinforcement that the UK lacks a lot of protections of basic human rights, though.
And there are plenty of people hell bent on removing those that remain.
Source: worked in that industry for almost 20 years.
When laws are changed with careful consideration, for both individual rights and social good – well, that's the very essence of the social contract.
It is a valid argument where I used it - against your purported bewilderment.
That's not the right under discussion.
The right is, in your own words: "the right to make photorealistic sexual material of their unconsenting obsession".
Your exaggeration of the scope does your argument no favour.
This is one of those "I never agreed until it happened to me" type things.