Hacker News Reader

Comment by Y_Y | Hacker News Reader

> Laura Farris, the minister for victims and safeguarding, said the creation of deepfake sexual images was “unacceptable irrespective of whether the image is shared”.

This is an interesting one because it violates the vague principle of "whatever done by consenting adults in the privacy of their own home is cool".

Like what if someone pastes my face ona naked body, or imagines me naked, or has a dream about me flying through the window naked and writes it in their private dream diary.

I think it's getting into hairy legal territory if the "infringing material" is only seen by the otherwise innocent creator on an airgapped computer.

View on HN

1967, then Canadian PM Pierre Trudeau on the government and consenting adults in their bedrooms: https://www.youtube.com/watch?v=6OoqgGgmVmI

I think the issue here is the between consenting adults part, deepfakes in this instance are probably not between consenting adults. I also think it's important to think about the reality of how this likely plays out. If someone deepfakes me and faps and I never find out and they never tell anyone, it would be quite hard for there to be action. So if someone is deepfaking you and you know about it, there must be a logical path there, and none of them look good for the deepfaker. Can't say I know where I land on the issue, and I've thought about it for days and hours.

Except the actual human being who resembles the deepfake image isn't actually involved at all.

People are acting like a "deepfake of a person" is something different than a drawing, painting, song, poem or any other artistic creation inspired by, based on, or with elements drawn from a real person.

"If someone is deepfaking you" is such a weird imaginary problem. Someone might be imagining you. Someone might see you in a coffee shop while writing a screenplay and use the outfit you're wearing to describe a character in their story. Who cares?

If someone is committing fraud by trying to pass off a deepfake as a real photo, then we already have laws around fraud, defamation, harassment to cover that. If someone makes a deepfake that looks similar to a real person for their own amusement, that's no different than writing fan-fiction or sketching in a notebook.

You should think long and hard about the second- and third-order effects.

No such thing.

Distribution of an image is not an effect of its creation.

Then you've clearly failed to think about those downstream effects. It's okay, a lot of people – not just in tech – find this challenging.

I think it would be better if you gave your views on what the downstream effects are, instead of just saying that other people can't think of them.

> deepfakes in this instance are probably not between consenting adults.

Deepfakes in this instance are not /between/ anyone. Only one person even knows of it.

Creepshots are not /between/ anyone. Only one person even knows about it.

For sure but for the otherwise innocent creator on an airgapped pc there is nothing to worry about. Who would ever know?

I do think the other examples you gave have one significant and fundamental difference - no one would mistake those things as being real. The problem with deepfakes is these have a similar social impact and capcity for harm as real nudes do, only there is no element of consent. You could ruin someones life quite easily with deep fakes, but not with your lewd poetry or sketches. So it seems reasonable to me to ban these with a blanket ban.

I would prefer we lived in a world where that wasnt neccesary, but for every technology you must imagine the most sinister and exploitative application and realise that someone will try that and worse. How do you even begin to combat deepfakes and the social problems they bring? how many bodies are considered acceptable until we get it right? Possibly, its best to over reach a little then relax the grip rather than to take it slow, given what is on the line for some.

I think the point is about the images which will never be shared No one wonders why they shouldn't spread fake nude photos of real people

and again, if they will never be shared, there is not much to worry about. We do need tools to persecute people who failed on the 'never be shared' front. Be it theft or otherwise, if others get the material from them, it doesn't matter, it should be treated as guilty of producing with intent to distribute. This is for two reasons, the first being this way people cannot hide behind the 'oopsie' defense.Second, if you made compromising material of others without consent, your duty of care is through the roof. If it leaks, you are at fault, no matter how hard you tried to prevent it. You never had to make that material. There was no reason to make it other than a selfish self-centered personal pursuits of a hedonistic nature, now if its victimless I don't care personally but the moment your selfish self-centered behavior hurts others, you are a danger to society.

Don't you think that's impractical? All attempts to stop people from using the Internet to do things on their own have failed. If the creation of such images is considered a crime in itself, then it will be obscured. Therefore, treating dissemination as a crime is not only logical, but also practical. And when it comes to accidental leaks, even if the person who created the image didn't pay attention, the leak happens by somebody anyway. I think it would be more practical and lead a better result to punish those responsible for the leak itself

>You could ruin someones life quite easily with deep fakes, but not with your lewd poetry or sketches. So it seems reasonable to me to ban these with a blanket ban.

A ban won't stop bad people from making them anyway. On the other hand, proliferation of deepfakes would make the existence of certain real or fake photos insignificant, as everyone would accept that anyone could make them easily.

> Under the legislation, anyone who creates such an image without consent will face a criminal record and an unlimited fine.

I feel like "without consent" to be pretty key here.

You don't have a right to consent to being part of an artwork. For example, even the King of England could not walk into a museum and demand they take a painting down because it's unflattering.

And now you do have the right to consent to being in porn.

Unless the King of England petitioned for law that gave him that right.

Which is basically what we're seeing here.

And that would be an authoritarian measure.

To petition change isn't authoritarian. That's the foundation of democracy.

The petitioning is not authoritarian but the measure is.

In that sense, all laws are.

Actually, you do.

IF the artwork is porn.

In fact, if you're not old enough, it's just flat out illegal. You'll go to prison for making that artwork. And you arguing that it's "art" is not going to find much resonance in US courts at least. You're likely to fail the Miller test if US citizens are the judges.

The basic obvious issue is “can you draw porn of someone”.

The implied answer here appears to be “no” and the reason this is deemed necessary is an emergent effect of scale of being able to do this at such volume and quality.

Saying you can’t draw porn of people without regard to implied authenticity or medium seems somehow more reasonable to me.

There just seems to be so much gray area here. Where is the burden of proof? Can someone argue that the depiction is of a fictional person who just looks similar to Plaintiff? How similar is too similar? Who's the judge of similarity? What if the training data did not contain any pictures of Plaintiff and the results were just tuned until it looked like him/her? What if Plaintiff has a twin sibling or someone who looks just like him/her and that person consented? What a mess--who's going to litigate all of this?

I don’t feel like anything is implying similarity is sufficient to make an artwork criminal.

Although I might call out the opposite problem of making it a crime to request a sfw picture of a specific person and have the model generate something porny

There's so much gray area because some people desperately wants porn banned than abuses and harms punished, and clarifying gray areas all the way removes side effects on purely generative porn.

Personally I think head-swap porns should be handled with defamation logic than general anti-porn laws. I find it more likely that I would be disgusted by deepfake porn of mine if I were to become such public figure that warrants it, than appreciating it, but it feels weirder that local police would be deciding whether am I to like it or not.

This is not a blanket ban on generated porn.

“It’s essential that the police and prosecutors are equipped with the training and tools required to rigorously enforce these laws in order to stop perpetrators from acting with impunity.”

We don't even have a tool that could reliably ID the subject of such a processed visual depiction.

> Who's the judge of similarity?

The judge.

https://en.wikipedia.org/wiki/I_know_it_when_I_see_it

The justice system is a computer made out of humans executing written programs called laws. As we've seen from decades of computer vision research, writing out an image classifier in a finite set of symbols is intractable. So instead the justice system queries an oracle (a human) to do the image classification.

Maybe in the future there will be a way to compile a learned image classifier into a set of directions in English and we will have laws that are followed by pages of what is essentially a small artificial neural network represented as an English flowchart, but I doubt it, that sounds dumber than asking a person to do it. You just have to make sure you're picking the right person to do the deciding.

Because then people can and will create fine tunes of models trained on specific people and just share those instead. I'm worried we will soon hear about a girl bullied into suicide because of a model she was powerless to stop from spreading.

It'll be about two seconds before creation of such models is declared an illegal act if a few suicides happen.

sigh.

I file this whole issue under: "This is why we can't have nice things".

People who create models like that ruin everything for everyone.

> Laura Farris, the minister for victims and safeguarding, said…

Instead of looking for those answers in the text of Farris’ summary of the law, we should look at the text of the law itself.

Edit: https://bills.parliament.uk/bills/3511/publications probably has the text we're looking for, somewhere.

The implication is that only one party has consented when these fakes are created, and therefore not all involved have given their consent.

Courts have previously deemed cartoon imagery of children to be equivalent to a real child, so considering the deepfake to be a real person that needs to give consent is not without precedent.

There's precedent for that already, it is not blazing any new territory. Mere possession of CSAM is very illegal.

The existence of a CSAM implies a crime was committed though, since there is no way to legally produce such an image. Possessing a CSAM image is pretty much the same as aiding/abetting the original crime.

A private deepfake image is not the same.

That's not true, the justification outlawing CSAM being the result of a crime is not the actual legal justification for banning it and it would not at all hold up in court.

There are plenty of incredibly disturbing videos and images of some of the most gruesome crimes committed and they are lawfully accessible and obtainable on the open Internet, including some such images on Wikipedia. The documenting of a crime is not and has never been illegal.

Per jjgreen’s comment, in UK law, for CSAM, an actual child need not be involved. Same logic proposed for non-consensual deepfakes.

You might be right, based on Kranar's comment. I thought that CSAM was illegal to artificially create, as well, which would be similar to a deepfake.

On a related note, I assume there is already a lot of deepfake CSAM in certain circles. If that is ruled legal, it will make finding and prosecuting real life CSAM creators much more difficult.

At least in the US.

Depends. Some jurisdictions classify CSAM by end results, others by process. Where it's process oriented it's not CSAM unless a child is not physically abused.

> The existence of a CSAM implies a crime was committed though, since there is no way to legally produce such an image

Well if those images are illegal, you're right that there's no way to produce them legally, but that's because they are on their face illegal.

If you mean there is no way to produce CSAM that doesn't break other laws, I think you're wrong - e.g. see the case of teenagers taking nude pictures of themselves. That is an otherwise legal thing to do, but having those images is illegal, as I understand it, and you can search Google for many examples of teens prosecuted for sexting images of themselves. I don't know if anyone has been prosecuted for merely having nude images of themselves, but that might just be because no one would even know about it, necessarily.

You are certainly right and there is good reason for that, however, it's worth noting that in the U.S., at least as per Ashcroft v. Free Speech Coalition [1], simulated CSAM is legal.

https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...

Thanks for the cite; I was very sure that simulated CSAM was just as illegal as the real deal, so I appreciate the correction.

I wonder if deepfake technology is going to change our stance on that. With enough motivation Congress could take it out of the hands of SCOTUS.

Did you completely miss this section?

https://en.m.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coal...

No, I was careful to not make any absolutist claim and even provided a reference for people to read further. Pornography in general is subject to the Miller test, so it should not be surprising that simulated CSAM is also subject to the Miller test as well.

"simulated CSAM" is a strange term - "CSAM" itself could be read as a dysphemism of CP whose purpose is less to emphasize the heinous nature of the abuse involved, and more to defend "ordinary" pornography as licit by putting a semantic distance between it and child porn. But of course if it is simulated, there is no abuse going on so the term becomes meaningless.

> "CSAM" itself could be read as a dysphemism of CP whose purpose is less to emphasize the heinous nature of the abuse involved

Maybe it could be read that way by some, but that's not why the term was adopted and it's not why the term is advocated for by anti-sexual violence organizations.

https://inhope.org/EN/articles/child-sexual-abuse-material

https://www.rainn.org/news/what-child-sexual-abuse-material-...

https://www.thorn.org/blog/decoding-the-language-of-child-se...

As for the fact that making it simulated renders the term meaningless, the entire point of a simulation is that it is not the actual thing that is happening but rather an imitative representation of something that could be real. Presumably the abuse towards a child could be real, and hence it's possible to imitate that abuse in various forms of media. It is that imitation that constitutes the simulation.

If you argue that it makes the term meaningless, then you must accept that the simulation of anything is meaningless.

> If you argue that it makes the term meaningless, then you must accept that the simulation of anything is meaningless.

I think that what is being argued. Put 'simulated' in front of any legal term and it becomes meaningless for its legal usage. 'Simulated larceny' is meaningless. 'Simulated murder' is meaningless.

Yeah, good luck simulating a terrorist attack at an airport, or simulating legal tender, or simulating a police officer.

There are plenty of laws that make it an offense to perform a simulation. Nitpicking the term simulation as being meaningless, whether in a legal context or otherwise, isn't a particularly noteworthy discussion to have. I think most people understand the meaning of simulated CSAM and can have an opinion on whether it should be legal or illegal without being pedantic about the word "simulated".

CSAM means something specific. It means 'a child was abused, here is evidence'. A simulation of that would be training materials for LEA or forensics teams to practice their craft.

You are really stretching here, and it is not 'pedantic' to ask that specific terms have specific meanings especially when it is a legal issue.

>A simulation of that would be training materials for LEA or forensics teams to practice their craft.

You are laughably misinformed on this matter in a way that is quite incomprehensible. You are welcome to refer to the legal definition as per the actual legal text instead of continuing to make rather bizarre claims on a topic where it's clear you know nothing about:

https://www.congress.gov/bill/108th-congress/senate-bill/151...

Even without having to refer to the actual text, I am confident most reasonable minded people understand the meaning of the term "simulated" and can take a position on whether it should be legal or not without arguing whether simulation means training materials or whether the term is legally meaningless.

The fact that you don't is all I need to know that continuing this conversation with you is entirely pointless.

For what it’s worth that is a bad vague principle

>"whatever done by consenting adults in the privacy of their own home is cool".

For example, cooking meth.

Reply on news.ycombinator.com