In Spain, girls report AI-generated nude photos of them being shared at school
english.elpais.com
english.elpais.com
There are potentially a couple of positive social outcomes that may emerge from this trend though.
1. everyone who sees a deepfake of themselves will be forever viscerally skeptical of any claim relying only on photos and videos socially shared online.
2. everyone who sees actual nude photos or videos of people they know will be skeptical they are real, thus taking away the power of revenge porn or ransom porn.
This is just casually dropped as though a photo-realistic AI image == an amateur painting, in production quality, in required effort and time, as though they're even remotely comparable. Which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy.
> It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.
I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels.
Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world.
And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.
You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings, just like you can outlaw pictures of religious imagery because they hurt feelings, but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy.
Giving up the right to online anonymity is a non starter as it would make it impossible for journalists and dissidents to communicate freely which is a requirement for democracy to function.
This statement is hilarious because we cannot even do the former for schoolchildren, let alone the latter. Between casual bullying resulting in fatalities due to the utter disinterest of the administration at play, to school shootings where police cower outside while someone murders children en masse.
Apparently being a child now is just grounds to have people make pornography of you, bully you relentlessly, with the distinct possibility you might die in a hail of gunfire one day for literally no reason.
No wonder the birth rate has cratered.
> You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings
CAN and SHOULD. INCREDIBLY illegal. Like registering the creator as a sex offender illegal, because that's what this is. It's a sex crime.
> but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy.
So we just give up?
That said, in a global internet any technology that can be created, will be crated, and made free and open for anyone to use. We can never escape this fact, and can only educate people on how to protect themselves as individuals as much as possible.
Google, Facebook, and modern surveillance capitalism app culture is -optional-. Smartphones have turned into a net negative that make most people miserable for instance. I for one do not have a smartphone, and am happier for it.
Maybe, but that's not an excuse to do it, like: "I do it because anyway someone else will". Silicon Valley is throwing a lot of money into stuff that "someone else" would do, and it is not clear to me that this someone else would always have that kind of resources.
I think it should be legal to burn flags. I would also never do it or advise anyone to do it.
Sometimes a free country requires allowing assholes to exist, and educating each other on how to ignore them.
I'm glad you asked!
Step 1: Anyone who uses any media, be it watercolors or generative AI, to create pornographic content of a minor should go to a LOT of prison and be registered as a sex offender.
Step 2: Educate children on these tools, what they may be used for, what they may not be used for, and illustrate vividly the extreme consequences thereof.
Step 3: When it inevitably happens anyway, enforce those consequences. The offender needs to go away for a long time, and be subject to rehabilitation programs to re-enter society. They should be treated as any other sex offender would, because that's what they are. Their age does not change their crime.
Step 3.5: Support the victim. Make it abundantly clear that anyone possessing this material any further will be subject to less severe certainly, but still severe consequences. It is to be deleted, to the maximum extent possible, from the maximum number of places possible.
Step 4: Any platform that permits this content to be distributed upon it should also be subjected to extremely steep fines and if the issue persists, seizure by the appropriate authorities. And frankly this should be expanded to all CAM, regardless of origin. Don't wanna regulate it, or feel you can't enforce this? Then shut down. Nobody asked you to to build a website and if you can't keep CP off your website, then your website doesn't need to exist, full stop.
Step 5: At this exact moment, begin changing how these generative tools are constructed. Make sure they only use training materials that are properly licensed and given over by their subjects and creators. If a given AI contains the ability to generate pornographic material, then it's distribution should also, in turn, be regulated as pornography already is. Yes, that will be probably be just as ineffective as it is for regular pornography. No, that is not an excuse to not do it. A tool that can make pornography on demand is indistinguishable from pornography.
And yes I am certain all of this will stifle innovation in the AI space. I don't care. Frankly if this kills it off entirely I think that's an unmitigated win.
Do you really suggest we abolish the right to privacy and anonymous speech in order to support the type of universal censorship you propose?
No, we can't stop people sharing it over encrypted channels. What we can do is mandate that these AI include identification data in their generated images, which is tied to a singular person. And from there, we can identify and prosecute the creator, and through court orders to whatever platform holder is at play, get a list of everyone who downloaded it and shared it.
We do this shit constantly for major rights holders. We know how. We just choose not to because too many people think it's "just boys being boys" and enable a culture that thinks sex offenses aren't worth our time.
You talk of punishment in the context of physical harm (fist vs. face), while deepfakes concern mental distress at worst. 2nd order effects like reputation damage, lost income etc can be adressed by lawyers.
Yes mental distress can be severe to some people. Or (for example) cause a group of people to harass someone.
But a) such harassment is a separate act from posting a deepfake. And
b) If deepfake postings happen often enough, it kinda takes the sting out. Few people today would be deeply disturbed if someone posts their head shopped onto a pornstar's body. That's easy enough that anyone can do it, it's been done many times, so the newsworthiness / shock value is gone.
If you know the victim in person, you'd probably be able to tell whether some video is fake or not. Or ask that person. If victim is a random stranger & deepfake disappears into the 'internet background noise', who cares.
Either way, when creating a deepfake is easy enough, it simultaneously grants a victim plausible deniability.
So ehm... "horrifying" is too strong a word imho. Try "annoying", "bad taste" or "disrespectful". Punishment (if any) should fit that.
THAT is horrifying. It's horrifying both by the base actions at play and, far moreso IMO, that because someone in the tech space thought it was monetizable enough to warrant investment, it will happen. It did happen. That genie is now out if it's box and when confronted with the ethical dilemmas it seems most people in the engineering space are content to shrug their shoulders and go "shit happens."
That's horrifying. It's horrifying that all a technology has to do is demonstrate it's ability to make money, no matter how socially or culturally corrosive it might be, no matter how much harm it might do to individuals or even entire professions, no matter how unethical and just straight up creepy it is, doesn't matter. If it will make someone rich, it will be done, and it will be done with the best hardware and expertise money can buy, and that social order will be corroded, that culture will be corroded, those individuals will be harmed, that profession will be obliterated, and those checks will be cashed.
THAT'S horrifying.
The #2 cause of death for people in the US aged 10-14 is suicide. Suicide is #3 for ages 15-24.
> b) If deepfake postings happen often enough, it kinda takes the sting out.
Ask any child subject to chronic bullying whether the repeated abuse "takes the sting out".
At some point soon every teen will know how to generate pictures of anything they can imagine. Teach impacted teen girls how to make deepfakes too and that it is not real. The world is not ready for this, but it is happening and can not be stopped so education is all we have left.
It is only shocking right now because it is new and seems like magic.
We should be educating teens on body positivity, and that anyone that shares a deepfake of them is announcing of how big of a child they are, and why they should not be taken seriously. It will just become an eye roll once everyone can do it and understands how easy it is. Background noise.
terminally uncreative take. I'm actually curious what you mean by free society tho because clearly we are not totally without rules and taboos, what are we free to do and where do we draw the line?
imo deepfakes are not that different from revenge porn, it is an image of you being shared without your consent. doesn't matter that it's 90% synthesized, your image was used in a way designed to hurt you or at least your reputation. we have laws about libel and slander, where's the incompatibility?
What can you actually do to truly stop it? Ban deep fake software? How's that ban going to work out? To what degree is libel and slander present in current schools and how often is it successfully addressed?
At some point enforcement becomes impossible without killing access to a free anonymous internet entirely.
I am not saying that is okay, but I am saying it is impossible to stop without shutting down the internet.
If anything deepfake tech reduces the demand to abuse real humans to obtain some types of pictures, which might reduce harm, even if in gross way.
Also you can find copyright movies too just as easily no matter how many billions have been spent trying to stop it.
AI is taking off like a rocket. Soon it’s going to be as easy as asking chatGPT to generate the photos/videos/whatever you want - and while chatGPT may be censored, there’s almost certainly going to be open source models that are not censored
Particularly such behavior that in order to reduce the immediate harms it causes, would require reducing the freedom of society as a whole. A simple example, rather less sensational or revolting than pseudo-nude photos of schoolchildren but far more pervasive and destructive, and similarly immune to attack under today's laws and circumstances, would be noise pollution ordinances, which are seldom enforced but frequently abused. Such abuses degrade the environment in innumerable, but sub-actionable ways for hundreds of people so that one person can have something they want, ie straight pipes on a harley/whatever being gunned through residential streets at 2:00 a.m. in the morning.
I feel like this class of abusive behavior is on the rise (at least, in Spain as well as my country) and will cause a much more catastrophic breakdown of a free society if it remains unchecked, but that we have no means of curbing under our current frameworks without compromising those frameworks themselves. At the same time I'm really tired of this posture of conceding to assholes on the grounds that tjey effectively hold "free society" hostage as a shield for their individual need for validation and/or financial gain.
If it's about comparing generations, it's quite likely that your grandparents (or at least their parents) won't be here to suffer the consequences of modern society (I mean climate change and biodiversity loss), but you will.
So yeah, I'm not convinced that we have it better than a few generations ago.
Before going into enforcement, I would have talked about ethics :-). It feels like it is not too much to ask of people who can create new technology that they think about the consequences of their work.
These deflecting arguments amount to no more than a modern irreligious version of "the s/devil/technology made us do it"
Countries have tried this experiment like China and North Korea, and it seems to only serve those in power, not the public.
Let us say we did ban all open source decentralized technology in the US, and all websites that to not follow government rules, and mandate all digital communication in the US happen via government approved apps and devices.
How long do you think it will take before a US president with authoritarian ambitions decides to take advantage of this single point of failure?
Free anonymous and private communication is mandatory for a functional democracy. It has some bad side effects, but the side effects of not having it are most assuredly worse.
Out of curiosity: can you mention one democracy where people generally have "free anonymous and private communication"?
I can't think of a single democracy where journalists, dissidents, and whistle blowers are not able to communicate and coordinate or even transact with others anonymously and privately when needed, thanks to decentralized tools like encrypted email, matrix, tor, bitcoin, etc.
In fact sometimes when a given state oversteps, like Texas on abortion, it is -critical- that people can communicate anonymously to preserve basic human rights that other states still recognize.
If that was ever restricted we are in for a world of pain.
Most people and companies don't use encryption to protect themselves against governments, even though it would not be that hard (one could use ProtonMail for company e-mails, Signal/Threema/Matrix for messaging, etc, but instead everybody seems to be using GMail and Slack/Teams). Being anonymous is actually hard.
So yeah, the only thing that "somehow" prevents the US government from spying on everybody is law, because they clearly have had the capability for a long time. And given the occasional whistleblower, it feels like it's not actually enough.
Actually I would even wonder if access to fairly-anonymous and private communication is not more important in non-democracies. Because in a functioning democracy you should be able to discuss matters more freely (at least to some extent).
Anyway that's probably off topic, and I didn't mean to say that governments should ban encryption (encryption does exist, bad actors will always be able to use it even if you ban it). But that does not mean that governments should not interfere with technology in general, I would say.
Yeah not sure, maybe I'm completely off topic here.
1. allow encrypted anonymous communication and the ability to use that to defend democracy
2. disallow encrypted anonymous communication and the ability to get away with sharing deepfake nudes
Both of these are bad, but one is clearly much worse than the other.
Anyone calling for steps as drastic as internet censorship to stop one comparatively minor issue are clearly unaware of the consequences of that road.
This is especially relevant given that every decentralized system is actually an illusion, a merely temporary state of transition from one centralized regime to the next.
Well, not everybody has the resources to make everything. Pretty sure ChatGPT would not have been written (and trained) by a student in their room.
The open source community is actually going much faster with LLMs than the corpos, as usual. They can share information and collaborate while the corpos are too focused on competing.
It’s only this very narrow window of recent history when photos have existed and been broadly considered “proof” of reality while being difficult to forge. This time now appears to be coming to an end.
In this particular case the issue isn't even the technology it's parenting. The students generating these images are in their teens and are more than capable of understanding that using images of their classmates is not only immoral but also likely illegal. But when left to raise themselves using the iPad-Babysitter you end up with the moral free for all that we are seeing today. Until 18, parents should be held equally responsible for the actions of their kids.
This is a bad example, hollow points have a legitimate use in the prevention of collateral damage, among other things. Unless you mean it's an exception because it's clearly a good technology, but I'd also say that's not true. It falls under neutral.
Given most offensive weapons have a defensive purpose, both active and deterrent, their is a case for them being neutral.
A bio weapon, near shore detonated nuclear device, or other doomsday devices that are designed to indiscriminately kill large numbers of civilians strike me as evil tech.
I find it interesting how you contradict yourself in your very first sentence. "Technology is neutral, except when it isn't". To me it means that it is not neutral, doesn't it?
Deserts are dry places except for rare occasions, when they can even be flooded. No contradiction just how language works.
I did not mean to ask "Is technology sometimes neutral?", but rather "Is technology always neutral, and in this case, is this particular technology neutral?". I did not write all that, but I assumed that it would be inferred from the context. And that, too, is how language works.
Also note that my above reaction was not aggressive. I genuinely find it interesting that they think "technology is neutral" but start their explanation with a sentence that clearly shows that they think technology is not neutral.
In common language there is no difference between "Yes, almost always, except.." and "No, not 100% always, there are some very rare exceptions". That sounds contradictory at first because one starts with yes and one with no, but both say the same thing and people are perfectly fine with shortening "90% true" to true.
The reason for that is that your question is kinda trivial to answer and kills any discussion for no gain of insight. "Is A literally, always B with absolutely no exceptions?". No, unless you talk about mathematics you don't get absolute statements.
> The reason for that is that your question is kinda trivial to answer and kills any discussion for no gain of insight.
Is it, really? What is the trivial answer? Is technology neutral or not? You seem to acknowledge that there are exceptions, so... does that mean that you think that technology is trivially not neutral? Because let me tell you that it is absolutely not a large consensus (to the point where I am genuinely not sure if that is your opinion or not).
But assuming that you agree with me that technology is not neutral, then my point was: maybe those who worked on that technology should have thought more before doing it. And there I was expecting constructive comments like "the technology itself is really useful for [saving lives / helping biodiversity / put something nice here], and unfortunately some abuse it with deepfakes". Because if we all "kinda trivially" agree on the fact that the technology behind deepfakes is evil, then maybe we should do something about it. Like teach ethics to engineers.
In this case, I don't see what good comes from the technology that enables deepfakes, but I would love to be enlightened.
I still don't understand what your opinion is, though.
Ok, so some laws were adjusted around the use of small cameras. Yet there are still issues today. I mean, you have to actually catch someone using it. How many people are redlight scanning their airbnbs? Many people breaking the law just aren't getting caught. I fail to see how the laws really fixed anything. All that happened is people learned to live in this new reality and the news stopped reporting it because it's not new.
I'm not sure what you're getting at with the hollowpoint comment. In theory it could be applied for evil (murder) or could be applied for good (self defense, public defense, food, etc depending on one's beliefs about what is good vs evil).
20 years ago long before iPads when I was in high school there were definitely kids that would have done this exact same thing if the technology was available. Lets not pretend as though lose morals are a recent invention.
Deep fakes aren't the issue here. This same thing could have been done before AI with Photoshop or even scissors and glue.
If any technology is at fault here, it's social media, because that's likely where whoever is making these is finding the pictures of the girls to feed the AI.
That's probably a valid argument, but you should elaborate. What does this particular technology bring to society that balances for deep fakes that can destroy someone's life?
Cryptography can protect people, which sounds like a good thing, but it can protect bad actors, which is a bad thing. One could debate it, but overall it feels like cryptography is useful in many cases. On the other hand, I feel like we would be better off without e.g. blockchain or without e.g. SpaceX making space flights affordable for businesses in a world where the climate crisis may soon destroy us.
What about deep fakes then? What good does it bring that would balance the bad use-cases?
I have a feeling you're going to have a problem with any kind of "general" technology in the future. Or, lets change the initial premise this this Heinlein quote
>"Specialization is for insects," the science fiction author Robert A. Heinlein famously wrote. "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly."
The thing is with a generalized image creation software is it will do things you won't like. Be that representation of minors, or offending your king/glorious leader, religious sacrilege, or whatever it is you don't like.
Generalized software, much like humans, is capable of heresy.
I can agree to the argument that "everything can be used in a bad way". But that is not my point.
My point is: is any technology always worth existing? As a thought experiment, I can easily imagine a technology that would only be useful for torture, and I would consider it fundamentally evil. To the point where I would not want this technology to exist at all.
If you agree with that, then it means that it is possible to imagine a technology that should not exist. And consequently, it raises the question for every new technology: should this one exist, or not?
In real life, the ones that humans actually occupy true black and white situations almost never exist. Your last sentence will always fall afoul of "In my moral framework" and since moral frameworks differ between cultures and individuals you will always find disagreement on the question of worth. The idea of fundamental evil is not a scientific measurable concept.
To turn around your concept and reverse the danger. I see any non-Christian* as fundamentally evil and it is my moral and just duty to remove any non-believers from this planet in order to preserve the sanctity of what is good.
At best we can make a rule of laws at attempt to fairly balance the needs of others and attempt to deter actions that are so dangerous for society they cannot be allowed. But never believe that your rule of law is universal.
* replace with whatever religion/belief system suits your needs.
Of course, the question of whether a particular technology is good or evil requires a moral framework, that's a necessity from the "good or evil" part.
Now those who say "technology is neutral" imply - if I understand correctly - that technology is out of this discussion. They think that the use of technology can be good or bad, but the technology itself is neither.
And that is what I question: should we consider that technology is completely out of the moral debate, or not? Obviously if we conclude that it is part of the moral debate, then it depends on the moral framework. In this case, the technology behind deepfakes was developed by people that I am convinced don't approve of the use in the original article.
So it seems legit to me to ask whether those people questioned the technology they were developing, within their moral framework, before doing it and releasing it to the world.
Am I missing something?
I mean, it's a useless question of pedantry. It rarely changes the outcome.
I cannot say that I am a free speech absolutist, there are many people that write software that are, and in addition believe that software is speech. Which particular rules of this framework do you want to apply? I think greedy people that value money above all else are terrible, but they exist and create software regardless of your abohorance of doing so.
That would be assuming that the people who built the technology bringing deepfakes are such people, which I would question.
Maybe they just don't think about the consequences of their work. Maybe there is something to teach to engineers about ethics. I'm saying that because I feel like personally, I could have use some ethics class. Maybe it would have given me some keys when thinking about all that.
Is technology ever neutral by those standards? Seems like anything that has legitimate uses could be used for evil too.
Ok, let's assume that anything that has legitimate uses can be used for evil. But what if the technology is mostly useful for evil? Would you call it neutral?
Your example of a torture machine would depend on if you consider any scenarios where torture would be justified/good/useful. If it does have legitimate uses, then there should be no problem creating it and trying to limit its use. This is where your analogy breaks down though. You can't just cp/ctrl-c a torture machine like you can data/software. Restricting software and its use becomes much more difficult when we talk about stuff like deep fakes while trying to balance liberty.
It does make a difference: a machine that is mostly useful for evil should probably not be built, whereas a machine that is mostly useful for good things but is mostly used for evil... well... that's a people problem.
> Your example of a torture machine would depend on if you consider any scenarios where torture would be justified/good/useful.
Let's consider a democracy where civilians decide to build a torture machine - the conditions behind the deepfake technology ;-).
> Restricting software and its use becomes much more difficult when we talk about stuff like deep fakes while trying to balance liberty.
Note that I never mentioned restricting it. I mentioned not building it. I find that engineers tend to hide behind "technology is neutral" and "it is not illegal". But morals and ethics do exist for engineers too.
It was totally legal to build the technology to make deepfakes. It doesn't mean that it was ethical.
We're not debating if it was ethical, only if it was neutral. Technology is applied by people so it's always a people problem. If there exists a state where it is good and a state where it is evil, then it's neutral. If it has only one state, them it is not neutral (either it's good or bad; either of those is rare).
Is that a formal definition coming from an official source, or is it your personal opinion?
You seem to say "good = 1, neutral = 0, bad = -1, hence good + bad = neutral". What if a technology has a "good" score of 1 and a "bad" score of -1000? Does "good + bad" make it neutral then?
You can look up neutral in the dictionary - if it doesn't belong to one side or the other then its neutral. Better to think of it as a 0-1 probability scale. If you're a 0 or a 1, then you belong to one side or the other. Anything in-between is neutral since its not a certainty nor exclusively aligned with one side or the other.
My initial point was that IMO, we should always think about the impact a technology may have on society before we build it. I am not completely sure what the technology behind deepfakes brings to society, but I clearly see serious problems. If a technology makes significantly more harm than good, maybe it should not be built in the first place.
Maybe I need another word to make you understand my point, because "neutral" clearly did not work.