Hacker News Reader

Comment by lrvick | Hacker News Reader

It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.

There are potentially a couple of positive social outcomes that may emerge from this trend though.

1. everyone who sees a deepfake of themselves will be forever viscerally skeptical of any claim relying only on photos and videos socially shared online.

2. everyone who sees actual nude photos or videos of people they know will be skeptical they are real, thus taking away the power of revenge porn or ransom porn.

View on HN

It would not surprise me to find that in 30 years or so a new category of harm is codified into law which amounts to official and social condemnation of behavior that abuses the freedom of a free society. Or to put another way, behavior that pathologically elevates personal or individual freedom ahead of the net freedom of the society that forms an indiv9dual's supporting environment.

Particularly such behavior that in order to reduce the immediate harms it causes, would require reducing the freedom of society as a whole. A simple example, rather less sensational or revolting than pseudo-nude photos of schoolchildren but far more pervasive and destructive, and similarly immune to attack under today's laws and circumstances, would be noise pollution ordinances, which are seldom enforced but frequently abused. Such abuses degrade the environment in innumerable, but sub-actionable ways for hundreds of people so that one person can have something they want, ie straight pipes on a harley/whatever being gunned through residential streets at 2:00 a.m. in the morning.

I feel like this class of abusive behavior is on the rise (at least, in Spain as well as my country) and will cause a much more catastrophic breakdown of a free society if it remains unchecked, but that we have no means of curbing under our current frameworks without compromising those frameworks themselves. At the same time I'm really tired of this posture of conceding to assholes on the grounds that tjey effectively hold "free society" hostage as a shield for their individual need for validation and/or financial gain.

For me personally I think it is less that this behaviour is on the rise, but that I’m getting older and it bothers me more. After hearing some stories about what it was like during my grandparents lifetime there were no bylaw officers or noise ordinances or pollution control, so I’m not convinced that we have it worse than a few generations ago.

> so I’m not convinced that we have it worse than a few generations ago.

If it's about comparing generations, it's quite likely that your grandparents (or at least their parents) won't be here to suffer the consequences of modern society (I mean climate change and biodiversity loss), but you will.

So yeah, I'm not convinced that we have it better than a few generations ago.

I don't think it will take 30 years. The problem is that it will be very hard to prosecute, it will have to concern possession rather than distribution (since you don't need to distribute the artifacts, you just need to be able to generate them).

In a way this isn’t actually a new thing. Throughout most of human history there was nothing like a photograph and it was always possible to fake paintings and other depictions of anything.

It’s only this very narrow window of recent history when photos have existed and been broadly considered “proof” of reality while being difficult to forge. This time now appears to be coming to an end.

I'm reminded of the aliens on Galaxy Quest who had no conception of lying, and were all but unable to identify or understand it. Rationally, we may know that pictures and videos are not to be trusted, but we will never stop getting taken in, like an optical illusion.

> It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.

terminally uncreative take. I'm actually curious what you mean by free society tho because clearly we are not totally without rules and taboos, what are we free to do and where do we draw the line?

imo deepfakes are not that different from revenge porn, it is an image of you being shared without your consent. doesn't matter that it's 90% synthesized, your image was used in a way designed to hurt you or at least your reputation. we have laws about libel and slander, where's the incompatibility?

There's a difference between between actually stopping it and laws. Sure, the law can be there but it's going to be hard to enforce and we need to see how the case law pans out.

What can you actually do to truly stop it? Ban deep fake software? How's that ban going to work out? To what degree is libel and slander present in current schools and how often is it successfully addressed?

The incompatibility lies with it being unenforceable at scale even if illegal. Religious blasphemy is not legal in many countries. Sharing pirated content is not legal in many countries, but we all are on the same internet. Trying to lock up millions of people anonymously sharing data globally that is not even universally illegal is an absurd enforcement problem.

At some point enforcement becomes impossible without killing access to a free anonymous internet entirely.

Right but we're able to relatively effectively scrub child porn from the Internet right? What makes deepfake porn different?

Child porn is a couple clicks away for anyone with internet access. Any teen boy that wants to see pictures of similarly aged girls, finds them quickly. Remember that such pictures, no matter how gross, are legal in many countries who share the internet with us.

I am not saying that is okay, but I am saying it is impossible to stop without shutting down the internet.

If anything deepfake tech reduces the demand to abuse real humans to obtain some types of pictures, which might reduce harm, even if in gross way.

Also you can find copyright movies too just as easily no matter how many billions have been spent trying to stop it.

I think stopping sharing is possible it’s stopping generation that’s probably incompatible

AI is taking off like a rocket. Soon it’s going to be as easy as asking chatGPT to generate the photos/videos/whatever you want - and while chatGPT may be censored, there’s almost certainly going to be open source models that are not censored

> but the types of measures it would take to stop it are incompatible with a free society.

Before going into enforcement, I would have talked about ethics :-). It feels like it is not too much to ask of people who can create new technology that they think about the consequences of their work.

yep. I'm getting really weary of these shallow protests about protecting anonymity and "technology" above all else, that are grounded in some sort of vague idea that technology is beyond moral reproach or has a mind of its own, and isn't in fact just an extension of ordinary, and in the case of this article, mean-spirited and cruelty-motivated human intent.

These deflecting arguments amount to no more than a modern irreligious version of "the s/devil/technology made us do it"

You seem to imply we should have centralized government mandated control over all technology that allows online communication.

Countries have tried this experiment like China and North Korea, and it seems to only serve those in power, not the public.

Let us say we did ban all open source decentralized technology in the US, and all websites that to not follow government rules, and mandate all digital communication in the US happen via government approved apps and devices.

How long do you think it will take before a US president with authoritarian ambitions decides to take advantage of this single point of failure?

Free anonymous and private communication is mandatory for a functional democracy. It has some bad side effects, but the side effects of not having it are most assuredly worse.

> Free anonymous and private communication is mandatory for a functional democracy.

Out of curiosity: can you mention one democracy where people generally have "free anonymous and private communication"?

The US? All of them?

I can't think of a single democracy where journalists, dissidents, and whistle blowers are not able to communicate and coordinate or even transact with others anonymously and privately when needed, thanks to decentralized tools like encrypted email, matrix, tor, bitcoin, etc.

In fact sometimes when a given state oversteps, like Texas on abortion, it is -critical- that people can communicate anonymously to preserve basic human rights that other states still recognize.

If that was ever restricted we are in for a world of pain.

Sorry, I'm actually not sure what your initial point was. I guess I just reacted to "Free anonymous and private communication is mandatory for a functional democracy".

Most people and companies don't use encryption to protect themselves against governments, even though it would not be that hard (one could use ProtonMail for company e-mails, Signal/Threema/Matrix for messaging, etc, but instead everybody seems to be using GMail and Slack/Teams). Being anonymous is actually hard.

So yeah, the only thing that "somehow" prevents the US government from spying on everybody is law, because they clearly have had the capability for a long time. And given the occasional whistleblower, it feels like it's not actually enough.

Actually I would even wonder if access to fairly-anonymous and private communication is not more important in non-democracies. Because in a functioning democracy you should be able to discuss matters more freely (at least to some extent).

Anyway that's probably off topic, and I didn't mean to say that governments should ban encryption (encryption does exist, bad actors will always be able to use it even if you ban it). But that does not mean that governments should not interfere with technology in general, I would say.

Yeah not sure, maybe I'm completely off topic here.

The point on topic is that people must pick one:

1. allow encrypted anonymous communication and the ability to use that to defend democracy

2. disallow encrypted anonymous communication and the ability to get away with sharing deepfake nudes

Both of these are bad, but one is clearly much worse than the other.

Anyone calling for steps as drastic as internet censorship to stop one comparatively minor issue are clearly unaware of the consequences of that road.

Or the other option which is the much less glamorous route of improving the visibility, accountability and transparency of centralized control, rather than writing it off as a lost cause from first principles.

This is especially relevant given that every decentralized system is actually an illusion, a merely temporary state of transition from one centralized regime to the next.

Millions of people can program, and it only takes one to anonymously release code that anyone can use. Code is free speech. We -should- teach ethics, but we must also live in the reality that any code, text, or media that can be imagined will exist now. This genie is not going back in the bottle.

> Millions of people can program

Well, not everybody has the resources to make everything. Pretty sure ChatGPT would not have been written (and trained) by a student in their room.

see leaked paper from google "there is no moat".

The open source community is actually going much faster with LLMs than the corpos, as usual. They can share information and collaborate while the corpos are too focused on competing.

> It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination.

This is just casually dropped as though a photo-realistic AI image == an amateur painting, in production quality, in required effort and time, as though they're even remotely comparable. Which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy.

> It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.

I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels.

Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world.

And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.

Stopping someone from physically harming someone else can come with severe -punishment- as a deterrent and is much more serious than sharing -any- type of data online. Stopping someone from making a digital parody of someone else gets closer to free speech territory and is a lot more complicated.

You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings, just like you can outlaw pictures of religious imagery because they hurt feelings, but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy.

Giving up the right to online anonymity is a non starter as it would make it impossible for journalists and dissidents to communicate freely which is a requirement for democracy to function.

> Stopping someone from physically harming someone else can come with severe -punishment- as a deterrent and is much more serious than sharing -any- type of data online. Stopping someone from making a digital parody of someone else gets closer to free speech territory and is a lot more complicated.

This statement is hilarious because we cannot even do the former for schoolchildren, let alone the latter. Between casual bullying resulting in fatalities due to the utter disinterest of the administration at play, to school shootings where police cower outside while someone murders children en masse.

Apparently being a child now is just grounds to have people make pornography of you, bully you relentlessly, with the distinct possibility you might die in a hail of gunfire one day for literally no reason.

No wonder the birth rate has cratered.

> You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings

CAN and SHOULD. INCREDIBLY illegal. Like registering the creator as a sex offender illegal, because that's what this is. It's a sex crime.

> but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy.

So we just give up?

Unless you have any ideas for a solution, I suggest we focus on education to contextualize what kids will inevitably see online. Kids are going to see over 18 content be it deepfake or hardcore. Censorship approaches do not work.

And teach engineers about ethics, too? I feel like it could help, because they are the one making the technology and often getting a ton of money for it.

I do wish we taught engineering ethics, because I do not feel like the majority of silicon valley surveillance capitalism tech workers have any ethics beyond following antiquated legal requirements.

That said, in a global internet any technology that can be created, will be crated, and made free and open for anyone to use. We can never escape this fact, and can only educate people on how to protect themselves as individuals as much as possible.

Google, Facebook, and modern surveillance capitalism app culture is -optional-. Smartphones have turned into a net negative that make most people miserable for instance. I for one do not have a smartphone, and am happier for it.

> That said, in a global internet any technology that can be created, will be crated, and made free and open for anyone to use.

Maybe, but that's not an excuse to do it, like: "I do it because anyway someone else will". Silicon Valley is throwing a lot of money into stuff that "someone else" would do, and it is not clear to me that this someone else would always have that kind of resources.

Saying a bad thing should be legal, or asserting it is not enforceable, is not the same thing as recommending people do it.

I think it should be legal to burn flags. I would also never do it or advise anyone to do it.

Sometimes a free country requires allowing assholes to exist, and educating each other on how to ignore them.

> Unless you have any ideas for a solution

I'm glad you asked!

Step 1: Anyone who uses any media, be it watercolors or generative AI, to create pornographic content of a minor should go to a LOT of prison and be registered as a sex offender.

Step 2: Educate children on these tools, what they may be used for, what they may not be used for, and illustrate vividly the extreme consequences thereof.

Step 3: When it inevitably happens anyway, enforce those consequences. The offender needs to go away for a long time, and be subject to rehabilitation programs to re-enter society. They should be treated as any other sex offender would, because that's what they are. Their age does not change their crime.

Step 3.5: Support the victim. Make it abundantly clear that anyone possessing this material any further will be subject to less severe certainly, but still severe consequences. It is to be deleted, to the maximum extent possible, from the maximum number of places possible.

Step 4: Any platform that permits this content to be distributed upon it should also be subjected to extremely steep fines and if the issue persists, seizure by the appropriate authorities. And frankly this should be expanded to all CAM, regardless of origin. Don't wanna regulate it, or feel you can't enforce this? Then shut down. Nobody asked you to to build a website and if you can't keep CP off your website, then your website doesn't need to exist, full stop.

Step 5: At this exact moment, begin changing how these generative tools are constructed. Make sure they only use training materials that are properly licensed and given over by their subjects and creators. If a given AI contains the ability to generate pornographic material, then it's distribution should also, in turn, be regulated as pornography already is. Yes, that will be probably be just as ineffective as it is for regular pornography. No, that is not an excuse to not do it. A tool that can make pornography on demand is indistinguishable from pornography.

And yes I am certain all of this will stifle innovation in the AI space. I don't care. Frankly if this kills it off entirely I think that's an unmitigated win.

How do you stop people from making deepfakes and sharing them on decentralized or end to end encrypted chats that are easy for anyone to use? Facebook bans sharing copyright movies, and yet every teenager can share them anyway. Also doing this is even perfectly legal in some countries we share the internet with. It is -impossible- to to deploy universal censorship without shutting the internet down.

Do you really suggest we abolish the right to privacy and anonymous speech in order to support the type of universal censorship you propose?

I find it incredibly illuminating that upon asking me how we should go about doing the thing you challenged me to do, you engaged with none of it and instead descended even further into niche technology to try and make it seem like, because it's technically possible, we shouldn't even attempt to regulate it.

No, we can't stop people sharing it over encrypted channels. What we can do is mandate that these AI include identification data in their generated images, which is tied to a singular person. And from there, we can identify and prosecute the creator, and through court orders to whatever platform holder is at play, get a list of everyone who downloaded it and shared it.

We do this shit constantly for major rights holders. We know how. We just choose not to because too many people think it's "just boys being boys" and enable a culture that thinks sex offenses aren't worth our time.

(..) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying.

You talk of punishment in the context of physical harm (fist vs. face), while deepfakes concern mental distress at worst. 2nd order effects like reputation damage, lost income etc can be adressed by lawyers.

Yes mental distress can be severe to some people. Or (for example) cause a group of people to harass someone.

But a) such harassment is a separate act from posting a deepfake. And

b) If deepfake postings happen often enough, it kinda takes the sting out. Few people today would be deeply disturbed if someone posts their head shopped onto a pornstar's body. That's easy enough that anyone can do it, it's been done many times, so the newsworthiness / shock value is gone.

If you know the victim in person, you'd probably be able to tell whether some video is fake or not. Or ask that person. If victim is a random stranger & deepfake disappears into the 'internet background noise', who cares.

Either way, when creating a deepfake is easy enough, it simultaneously grants a victim plausible deniability.

So ehm... "horrifying" is too strong a word imho. Try "annoying", "bad taste" or "disrespectful". Punishment (if any) should fit that.

The blasé attitude of the people building this technology is what makes it horrifying. Literally the FIRST major objections I heard to generative AI/deepfakes was the strong possibility if not guarantee of how it would be utilized by bad actors, be they interpersonal or political in nature. And over and over the tech sphere has downplayed and ignored this reality in favor of the ephemeral "progress" which also happens, by sheer coincidence I'm sure, to be their paths to success both professionally and financially. And on this progress marched until we've reached a place where it is now feasible that a high schooler could take the likeness of a girl he likes, shove it into an image generator, and make porn out of her without her consent or knowledge and be totally free to distribute that however he sees fit.

THAT is horrifying. It's horrifying both by the base actions at play and, far moreso IMO, that because someone in the tech space thought it was monetizable enough to warrant investment, it will happen. It did happen. That genie is now out if it's box and when confronted with the ethical dilemmas it seems most people in the engineering space are content to shrug their shoulders and go "shit happens."

That's horrifying. It's horrifying that all a technology has to do is demonstrate it's ability to make money, no matter how socially or culturally corrosive it might be, no matter how much harm it might do to individuals or even entire professions, no matter how unethical and just straight up creepy it is, doesn't matter. If it will make someone rich, it will be done, and it will be done with the best hardware and expertise money can buy, and that social order will be corroded, that culture will be corroded, those individuals will be harmed, that profession will be obliterated, and those checks will be cashed.

THAT'S horrifying.

> You talk of punishment in the context of physical harm (fist vs. face), while deepfakes concern mental distress at worst. 2nd order effects like reputation damage, lost income etc can be adressed by lawyers.

The #2 cause of death for people in the US aged 10-14 is suicide. Suicide is #3 for ages 15-24.

> b) If deepfake postings happen often enough, it kinda takes the sting out.

Ask any child subject to chronic bullying whether the repeated abuse "takes the sting out".

"Your mom" jokes used to be something someone would start a fight over. Now they only elect an eye roll.

At some point soon every teen will know how to generate pictures of anything they can imagine. Teach impacted teen girls how to make deepfakes too and that it is not real. The world is not ready for this, but it is happening and can not be stopped so education is all we have left.

It is only shocking right now because it is new and seems like magic.

We should be educating teens on body positivity, and that anyone that shares a deepfake of them is announcing of how big of a child they are, and why they should not be taken seriously. It will just become an eye roll once everyone can do it and understands how easy it is. Background noise.

Reply on news.ycombinator.com