As AI porn generators get better, the stakes get higher
techcrunch.com
techcrunch.com
Following the link [0]:
> People viewing explicit images of you without your consent — whether those images are real or fake — is a form of sexual violence, said Kristen Zaleski, director of forensic mental health at Keck Human Rights Clinic at the University of Southern California. Victims are often met with judgment and confusion from their employers and communities, she said. For example, Zaleski said she’s already worked with a small-town schoolteacher who lost her job after parents learned about AI porn made in the teacher’s likeness without her consent.
> “The parents at the school didn’t understand how that could be possible,” Zaleski said. “They insisted they didn’t want their kids taught by her anymore.”
Holy shit that's fucked up. It wouldn't be right but I'd be hard pressed not to create deep-fakes of the people who fired me and/or the parents to show them how bullshit it is if I was fired for something I had no connection to and was actually the victim of.
[0] https://www.washingtonpost.com/technology/2023/02/13/ai-porn...
“The parents are complaining and we’re chickenshits” is almost certainly not an unlawful reason, if they even bother justifying the dismissal (which most employers don’t).
Factuality does not matter to a lot of people.
Curious if we change the language used if that changes how people think of it...
What if you imagine it? What if you describe it to your friend? What if you sketch it on your tablet and then delete it?
- most folks probably will agree that imagining sexual content with your teacher and keeping it to yourself is fine
- most folks probably will also agree that making photorealistic fake porn of your teacher and spreading it around is akin to defamation or sexual harassment
Let's take care of the extremes, then we can worry about the difficult middle.
I'm asking on purpose to figure out where the line is
Imagining it? That can't be distributed, so no issue here. Describing it to friend? I think it's no issue, as the setting is private, and words are ephemeral. Sketch it on tablet? As long as no one sees it, it's fine.
Now, let's spice these up with distribution. What if you describe it not to a friend, but to an audience at a stand-up club? Or in a tweet? Or in a written publication? What if your sketch gets photographed, and put online? What if it's a semi-known person, and a tabloid gets ahold of the photo?
I think the first and second scenarios differ a lot. And that while both can be likened to sexual violence, the second group is what does real damage.
That's because it largely is. It's also highly unenforceable, which raises the suspicion: how would a proponent intend to enforce it? By implementing surveillance? Or would this be just used to suppress political opponents for example?
>if you create something in a form which makes it possible to distribute, and then someone just happens to get access and distributes it, is it not your fault too? Here it becomes unclear.
I think if law would exist, then it would define these scenarios. I feel like there are a lot of already existing situations that are similar, and also legally defined. For example a legal firearm can be stolen from somebody, or accessed by a child, and some harm is done, then in some cases, the original owner is also implicated[0]. Or with drugs, possession and distribution are usually separated, even in the case of a completely banned substance, with distribution usually being a heavier category.
[0] https://www.criminaldefenselawyer.com/resources/can-i-be-pun...
>What you intend to do with the creation is what usually dictates the legal ramifications.
Disagree. Legal stuff is first concerned with what happened, and just secondarily with the why. This separates murder and manslaughter for example in the US, but in either case, the main thing is that someone died because of another person. If it was only intent, and no harm done, not even an attempt, there wouldn't be a case in the first place.
Now where the moral line is, that's another question, and intent indeed does play a huge part there. In Christianity for example, you can sin without even doing anything, as God is concerned not just with outside actions, but with a person's inner life too.
> Legal stuff is first concerned with what happened, and just secondarily with the why.
I guess it's more nuanced than we have space for here, lol. :) I was thinking of 'possession with intent to supply' for narcotics. The penalty is much greater for that than simple possession.
I think you are correct though, I'm mixing up legal and moral concern.
Most important information is textual, and there's really nothing at all that binds text to reality. So we have laws about fraud and libel for important matters. Trust is a hard problem! But we're working on it, and images were never much of a solution to it anyway.
Most of the time, we just don't care. Many stories are untrue; explicitly fictional. Some of them are untrue in ways that could emotionally damage others (gossip, insults). That's just a part of social life, to be mitigated socially, not judicially.
Images are going to be just like text. Let's all just embrace that already. You can't lie any more or less with them than you can text, once we all (inevitably) stop trusting them as though they were difficult to fabricate. There's a little turmoil coming, but there's less if we all just decide to get over it now rather than cling desperately to the idea that images are somehow more deeply intertwined with the real world than text is.
As for a world where we all learn to distrust images, I don't think we will fully get there. You still have people falling for email scams that by now are very clearly documented. Even a small percentage of people who are living in a different reality can be problematic for the rest of us.
* don't accept unsolicited messages or images from people you don't already know
* don't leave your corporate-moderated walled garden of choice to search for stuff on the open internet
* don't put too trust much into any information distributed primarily online
I think these things will all be necessary but they are far more dystopian than the internet that I once hoped for.
Rationally, we know all this. But biologically, we're wired to react first and think later; our reactions often shape our behavior more than logic. That's one reason that advertising and propaganda work; repetition trumps rationality. Add in the fact that large numbers of people are gullible fools who can be trained to uncritically repeat and spread untruths that appeal to confirmation bias.
It's not the abstract knowledge that something is possible that influences our gut reactions, its our experiences. When your friend group chat is chock-full of little jokingly-made fake images of each other, then your behavior of reacting to realistic photos as though they were real will dull and eventually go away entirely.
I'm not sure we can entirely 'trust' images/photos anyway right now. Even a 'genuine/undoctored' image, paired with partial/misleading info or outright lies has an impact on what we take from an image anyway.
People trust text and that's clearly written by someone not a snapshot of reality. There's a reason libel is a thing.
1) ignoring a problem and harms because "soon" we think such tricks may not work as well is a disservice to anyone unlucky enough to be harmed now
2) Photoshop and similar have been around for a long time now yet fake images and videos still often fool people on first glance, so it's a bit blindly optimistic to assume that in the future volume alone will change people's initial credulity
EDIT: elsewhere on HN today there was a link to an annonymous post on Blind claiming to be a Google employee on a PIP because of some sort of quota imposed on their manager. This may be true, it may not be. I don't have a way of judging personally, and obviously text on the internet has been full of bullshit for a long time. But is anyone here confident enough to claim that nobody who encountered that on HN or Blind would believe it as a result?
Interesting, this is the first I've heard of it. It does appear to be compelling, however.
The "deepfake" armageddon that's been blathered on and on about is obviously horseshit. If people can make a deepfake of anybody in five minutes, nobody is going to get fired (without getting the shit sued out of them) for deepfakes of them showing up. Also no one is going to believe in the pictures, even if they are real. This is a blessing to the future victims of revenge porn, one that people whose photographs were spread around before the dawn of generative AI won't be treated to.
This has always been a media panic to lay the groundwork for more censorship and more surveillance. We'll have to be licensed to run an algorithm if they have anything to do with it, and our models will require a Microsoft signing key.
Exactly. At some point people will just stop caring, which is the best of all possible outcomes.
It doesn't matter whether it's true or not. Many people will act as if it's true, either because they are fools or frauds. IF they don't get the result they want they'll claim the truth is being suppressed and demand inquiries and investigations. Regardless of the facts, such false claims will waste resources and cause personal reputational damage.
We live in a country where weather forecasters get death threats for mentioning climate change, hospitals get bomb threats for treating kids with gender dysphoria, and librarians are condemned as agents of Satan. Relying on collective rationality and technical awareness is foolish.
https://www.theguardian.com/us-news/2023/jul/16/chris-glonin...
https://www.cnn.com/2022/09/15/us/boston-hospital-bomb-threa...
https://www.latimes.com/politics/story/2023-01-27/school-lib...
The WaPo article cited that mentions the school teacher who was fired does _not_ mention any litigation. It seems like a failure to engage with the material if you're saying "no one is going to be impacted by humiliating deepfakes" in reporting that specifically mentions someone who was. Even if there was litigation, that puts a cost on the former employee which they may not be able to easily afford, and my understanding is that very specific criteria must be shown for most dismissed employees to get anything.
-----
edit:
I'll put it up as a service; the protection of women from misogynist school administrations.
We've advanced beyond when women would be fired from teaching for getting married or becoming pregnant, but not much.
Also, _all actual evidence is in the past_. You're trying to write yourself a permission slip to predict anything while ignoring all actual lived experience.
It's good that you understand the problem.
We've advanced beyond when women would be fired from teaching for getting married or becoming pregnant, but not much."
To be fair, we haven't heard of male teachers moonlighting as pornographers and getting fired for it. Until then, I would not call it misogynist...
I'm joking, but not really at the same time...
That teacher in the article - what if she was photoshopped instead of "deep faked" would anything have been different?
Photoshopping has been around forever.
Just a bit weird to me that people are predicting the AI end of the world with this stuff when its really just a slightly faster way to do something we've had for ages.
"Slightly" faster is vastly underselling it, deepfakes can be rendered in real time now. The same tech powers TikTok filters.
I can’t imagine reading “22 year old teacher gets fired for no fault of her own” and immediately writing up an explanation as to why nobody should be able to get in trouble for making and distributing deepfake porn of strangers in the future, but I am not so enlightened.
People believe far less convincing things now. People see After Effects flares and badly animated CGI on Youtube and photoshopped images and think aliens are real and cities appear in the skies over China. People will believe whatever confirms their biases, regardless of its quality.
I don't think this is going to change much, one way or the other.
Not only is is a bigger problem than a bit of puritanism, but it's one that doesn't completely solve itself.
Because there's no reasonable argument against it. What I want to understand is why people have such a low opinion of the intelligence of the people around them. I can't imagine a person so stupid that they would be aware that they can make a 1-click deepfake, but also be totally convinced by every photo they see. This would be somebody who runs away screaming when they see a train coming towards them on television (which people actually did at the dawn of film, but they got used to the new situation.)
> People believe far less convincing things now. People see After Effects flares and badly animated CGI on Youtube and photoshopped images and think aliens are real and cities appear in the skies over China. People will believe whatever confirms their biases, regardless of its quality.
People believe it because it is harder to do now. They use a very sensible heuristic (often badly) that nobody would go through the trouble to fake X images for Y reason. The question is explicitly about what happens when you don't have to go through any trouble to create convincing fake images.
If you don't want people fired for anonymous denunciations, you should concentrate on labor rights.
This is a good point. Nobody believes that any text is real and as a result nobody has been harmed by text since the advent of the typewriter (the point in history when it became difficult to ascertain the true writer of a piece of text). Surely images will follow in this vein.
We're literally commenting on a thread about a teacher being fired because parents are too stupid to understand that deepfakes aren't real.
> People believe it because it is harder to do now. Define harder.
Here's how to make a deepfake using stable diffusion (9 month old tutorial): https://www.youtube.com/watch?v=kJEHJXiwGVU
and here's how to train the model to make whoever you want given you have about 8 pictures: https://www.youtube.com/watch?v=9Nu5tUl2zQw
You don't even need your own gpu if you use Google Collabs, really. Hell, it requires such low grade hardware that you can do it with Collabs' free tier.
Honestly, because there are a lot of people out there like you. No matter how much evidence there is out there to the contrary, they'll still keep believing what they want to believe.
In a world we're we've gone through blatant political spoken lies having enormous consequences due to people believing them, books of fake text people believing, entirely absurd doctored images people believing some people some people are unable to believe that people will fall for faked videos.
As in regardless of what the evidence is, and how obviously it may point to a conclusion, some people will believe what they want to believe to maintain their world view - even if it's entirely counter-factual.
And whether their belief is "xyz people are lizard people", "aliens are real", or "people will never believe faked forms of media" they will continue to believe it no matter what is shown to them throughout their lifetime.
I think I’m the future when you see a digital image or video you’ll think the same thing: here comes some jerk to scam me.
From the article:
> The parents at the school didn’t understand how that could be possible
These people will vote on what they see.
Why are you expressing annoyance about people focusing on the "unimportant" problem of than dealing with the fallout of "AI fake porn generators" than about other people causing these problems in the first place by focusing on the problem of generating faked porn. There's no real importance to generating this stuff.
I don't know what happens to journalism and misinformation at that point, but it's not good. State level actors have been doing this already. India just recently got caught putting cheeky smiles on the faces of protestors after being brutalized by police to suggest they were faking their distress for their cause.
1) everyone in the entire world learns about how easily things can be faked as quickly as possible
and
2) everyone has perfectly rational Vulcan control over the emotions this stuff would provoke on first view?
Social engineering is already not that hard, take a minute and imagine how something like this could be presented as a late step as part of a multi-step con to really fuck someone over.
So why are we pretending that making it easier to make more visceral and convincing versions of that won't have any bad effects? We already know that there are people who will try to advance their own interests like that, in what world does giving them more tools make their job harder somehow? "A sucker is born every minute" was said pre-computers, after all.
This is not an argument that I see people making. They're saying that sending deepfakes to your job is just plain harassment, and that the photograph, which is only thought of as evidence because it is difficult to fake, will have lost its value as evidence.
The invention of photography didn't make it easier to fake narratives, it made it harder.
"We regret to inform you that our entire database of full body scans was accessed and downloaded by unauthorized persons. We're providing you with a free year of online reputation repair services. We apologize for any inconvenience."
If they're naive enough to actually believe that image, you can just reply with one showing them in bed with your spouse. That should settle the matter.
... in a world where it is understood that fake realistic images are easy to be made then this is not any more credible than a text without pictures.
Let's say your boss received an anonymous email accusing you of being a pedophile. What's the difference here in the world where a deepfake is no harder to make than an anonymous email?
An anonymous email will probably be a hell of a lot harder to accomplish than a deepfake, actually.
Here's the real scandal: techcrunch didn't blur out the nipples!
If people never the less want to be exhibitionists and do it, that’s fine.
Now, what’s true for real life should be true for synthetic productions. Depictions in real life that would be illegal should remain illegal in the synthetic world. Though I wonder if Japan will insist on pixelation of the groin.
Why? How is it any of your/our business what porn people generate and masturbate to in private?
What's your reasoning for this? Murder is illegal, yet depicted all the time in movies.
So is robbery, rape, any number of illegal activities. How do you propose to convince a population to make all those depictions illegal?
Or is it only sex that you have a hangup with?
I think creative works would be severely limited if nothing illegal could be depicted within them. If you're arguing only pornographic content should carry this restriction, I'd be interested as to why the same crime depicted in a non-pornographic film is better (in the sense that you think it should be allowed / not made illegal) than the same crime depicted in a pornographic film.
Even restricted to only pornography, I don't see why I (as a hypothetical creative) should not be allowed to include a description of something illegal within the context of a pornographic film. i.e. I think it should be (legally speaking) fine for me to make a hot a steamy porno that also includes, as a core part of the porno, a scathing critique of my local town's property tax implementation while a bank robbery is ongoing. If it matters, let's say the main characters / people having sex are also doing the bank robbing.
If you're trying to restrict what crimes are illegal to depict (rather than all crimes), then I'm not sure what metric you're using for that. Is it only explicitly sexual crimes that should not be in sexual films? Because there's plenty of crimes that are not intrinsicly sexual but that can be sexual because of the observer, e.g. Masochists can be aroused by illegal non-sexual physical abuse, would it be okay to have that illegal non-sexual physical abuse in an otherwise explicitly sexual pornographic film?
If you're concerned about people "getting off" to crimes, I don't think your restrictions are going to mitigate that at all. People are aroused by the abuse in Uncle Tom's Cabin[0]. If you're trying to prevent people from being aroused by "bad" things, I don't think you'll see much success. Hell is other people, after all.
[0]: I'm reading "The Masochistic Pleasures of Sentimental Literature" by Marianne Noble and it references a letter that's relevant here: "Even in my early childhood I loved to revel in ideas about the absolute master of one man over others. The thought of slavery had something exciting in it for me, alike whether from the standpoint of master or servant. That one man could possess, sell or whip another caused me intense excitement; and in reading 'Uncle Tom's Cabin' (which I read at about the beginning of puberty) I had erections."
Same letter, different reference, is quoted on page 65 here of "The Erotics of History" by Donald L. Donham: https://www.jstor.org/stable/10.1525/j.ctt2204r8x.12
Woof. Is it hot in here??