[flagged]
Grok is a platform that is enabling this en masse. If xAI can't bring in guardrails or limit who can access these capabilities, then they deserve what's coming to them.
Arguably morally reprehensible but it has not always been illegal (and still isn't in many places) if you're talking about images of adults.
This comment chain isn't about a difference of opinion, but a difference in morality - there's no debating possible about morals I think.
Who told you that? Go ask Pamela Anderson or Paris Hilton about that one. There are rules about material created without consent, but people do not retain a perpetual right to have formerly consentual material taken down. Hollywood, let alone the porn industry, would collapse overnight if every disgruntled star could have movies removed whenever they feel like it simply by withdrawing "consent" years after creation.
And for copyright, generally the person on camera is not holding the camera and so is not the creator/owner of the material. That is a regular issue where people attempt to use the dmca to remove images of themselves from websites.
She can't withdraw consent when she never gave it in the first place.
She claimes, later, to have not consented but given the sophistication of the production no reasonable person could believe she had no idea it was being filmed. She might not consent to the exact public release but she was certainly well aware of being filmed on the day. She consented to the creation.
Same issue decades later during the iphone hacks/leaks. They did not consent to public release, but did consent to creation and private distribution, sometimes even taking and initially sharing the photos themselves.
Let's start from the beginning, create and own:
You're sketching out some nude fanart on a piece of paper. You created that and own that. Thas has always been illegal?!
(This is apart from my feelings on Mechahitler/Grok, which aren't positive.)
Is twitter a piece of paper in your desk? No, it's not.
OP had "It has always been illegal and morally reprehensible to create, own, distribute or store "
It would make more sense then to instead say:
"It has always been illegal and morally reprehensible to distribute "
I am with you on publishing these...
What if I only thought about it? Still morally reprehensible? Or only if I tell others I think about them? Then you could argue it's sexual harassment.
Personally I think it's just embarrassing, not immoral.
Elites need a reckoning
> And internal comms could be asked for without the nuclear option of raiding the office
Relying on the accused to compile and provide evidence seems naive to me.
AFAIK police can just ask google/slack for the mails with proper warrant.
It sounds like they are following due process.
What a strange question.
Like any bad behaviour, the grown-up response should be discipline and education.
There's a million ways kids can misbehave. The idea is to get kids ready for the real world, not pretend there's nothing bad out there.
Obviously we don't want "point and click" AI nudes in the hands of minors, or kids having their own AI accounts in the first place. Parents and educators pay for their kid's devices and internet connections. If they're not being responsible, you take away the privilege until they learn about respectful behaviour.
If the kid is allowed to stay out after dark but ends up doing crime at those times, we don't ask the government to impose a curfew on every kid. We discipline the kids involved. And that's my last comment in this thread thank God, what a struggle.
Have we a outsourced all accountability for the crimes of humans to AI now?
Those boys absolutely should be held accountable. But I also don't think that Grok should be able to quickly and easily generate fake revenge porn for minors.
[1] https://www.nbcnewyork.com/news/national-international/girl-...
And the AI is at fault for this sentencing, not the school authorities/prosecutors/judges dishing justice? WTF.
How is this an AI problem and not a legal system problem?
You can’t just “undo” some girl being harassed by AI generated nude photos of her, so we…
Yes, we should have some protections or restrictions on what you can do.
You may not understand it, either because you aren’t a parent or maybe just not emotionally equipped to understand how serious this actually can be, but your lack of comprehension does not render it a non-issue.
Having schools play whack-a-mole after the photos are shared around is not a valid strategy. Never mind that schools primarily engage in teaching, not in investigation.
As AI-generated content gets less and less distinguishable from reality, these incidents will have far worse consequences and putting such power in the hands of adolescents who demonstrably don’t have sound judgment (hence why they lack many other rights that adults have) is not something most parents are comfortable with - and I doubt you’ll find many teachers, psychiatrists and so on who would support your approach either.
No, but if you send those people who made and distributed the AI nude of her to jail, these problems will virtually disappear overnight, because going to jail is a hugely effective deterrent for most people.
But if you don't directly prosecute the people doing it, and instead just ban Grok AI, then those people will just use other AI tools, outside of US jurisdiction, to do the same things and the problem persists.
And the issues keeps persisting, because nobody ever goes to jail. Everyone only gets a slap on the wrist, deflects accountability by blaming the AI, so the issue keeps persisting and more people end up getting hurt because those who do the evil are never held directly accountable.
Obviously Grok shouldn't be legally allowed to generate fakes nudes of actual kids, but in case such safeguards can and will be bypassed, that doesn't absolve the humans from being the ones knowingly breaking the law to achieve a nefarious goal.
Youths lack judgment, so they can’t vote, drink, drive, have sex or consent to adults.
A 14-year-old can’t be relied to understand the consequences of making nudes of some girl.
Beyond that, we regulate guns, speed limits and more according to principles like “your right to swing your fist ends at my nose”.
We do that not only because shoving kids into jails is something we want to avoid, but because regulating at the source of the problem is both more feasible AND heads off a lot of tragedy.
And again, you fail to acknowledge the investigative burden you put on society to discover who originated the photo after the fact, and the trauma to the victim.
If none of that computes for you, then I don’t know what to say except I don’t place the right to generate saucy images highly enough to swarm my already overworked police with requests to investigate who generated fake underage porn.
Teenagers do stupid shit all the time. But they still get prosecuted or convicted when they do crimes. They go to juvy or their parents get punished. Being 14 is not a get out of jail free card.
The problem stems from parents lack of parenting, a huge lack of real after-school programs, and the tiktokification of modern society.
30 years ago, we had a lot of the same "slap on the wrist" punishments because it was assumed when you got home your parent was going to beat your ass. That isn't a thing anymore (rightfully), because parenting through threat of violence just leads to those kids becoming violent parents.
Our problem is we never transitioned from violent parenting into any other kind. I watched my nieces and nephews get parented by YouTube and get social media accounts before they were 10. COVID created a society of chronically online children who don't know how to interact offline.
And yes, the tools to create bad shit are more accessible than ever, and I always come off as some angry gate keeper, but so much of the internet as it is today has become too easy to access by people incapable of the critical thinking required for safe use.
In the last 5 years, generative AI has taken over most of the "public facing" internet, and with internet literacy at the same level it was 20-30 years ago, we are back in the "walled garden" AOL era, but it is Facebook, Instagram, Twitter, TikTok that are the gardens.
I'm similarly repulsed by the idea of Grok generating images of kids, but if you draw a nude of an adult woman she's not going to get raped by that existing, and you don't have a right to not be embarrassed. Tough shit, deal with it.
In good faith, a few things - AI generated imagery and Photoshop are not the same. If someone can mail Adobe and a photo of a kid and ask for a modified one and Adobe sent it back, yes Adobe’s offices will be raided. That’s the equivalent here. It’s not a tool. It’s a service. You keep using AI, without taking a moment to give the “intelligence” any thought.
Yes, powerful people are always going to get by, as you say. And the laws & judicial system are for the masses. There is definitely unfairness in it. But that doesn’t change anything here - this is a separate conversation.
If not Grok then someone else will do it - is a defeatist argument that can only mean it can’t be controlled so don’t bother. This point is where you come across as a CSAM defender. Govt’s will/should do whatever they can to make society safe, even if it means playing whack a mole. Arguing that’s “not efficient” is frankly confusing. Judicial system is about fairness and not efficiency.
frankly, I think you understand all of this and maybe got tunnel visioned in your anger at the unfairness of people scapegoating technology for its failings. That’s the last thing I want to point out, raiding an office is taking action against the powerful people who build systems without accountability. They are not going to sit the model down and give a talking to. The intention is to identify the responsible party that allows this to happen.
Actually you'll see the opposite happen a lot - after Columbine, the number of school shootings went up [0] for example, because before people didn't consider it an option. Same with serial killers / copycats, and a bunch of other stuff.
Likewise, if it hadn't been in the news, a lot of people wouldn't have known you can / could create nudes of real people with Grok. News reporting on these things is its own kind of unfortunate marketing, and for every X people that are outraged about this, there will be some that are instead inspired and interested.
While a lot of punishments for crimes is indeed a deterrent, it doesn't always work. Also because in this case, it's relatively easy to avoid being found out (unlike school shootings).
[0] https://www.security.org/blog/a-timeline-of-school-shootings...
Yes, let's just jail every kid who makes a mistake, ya know, instead of the enablers who should know better as adults...
except for that one guy, let's put him in the white house
Did you see the part in the article about "raided in France" and "UK opens fresh investigation"?
You definitely can. You don't have to prosecute and send a million people to jail for making and distributing fake AI nudes, you just have to send a couple, and then the problem virtually goes away.
People underestimate how effective direct personal accountability is when it comes with harsh consequences like jail time. That's how you fix all issues in society and enforce law abiding behavior. You make the cost of the crime greater than the gains from it, then crucify some people in public to set an example for everyone else.
Do people like doing and paying their taxes? No, but they do it anyway. Why is that? Because THEY KNOW that otherwise they go to jail. Obviously the IRS and legal system don't have the capacity to send the whole country to jail if they were to stop paying taxes, but they send enough to jail in order for the majority of the population to not risk it and follow the law.
It's really that simple.
In this case, it's far simpler to prosecute the source.
The certainty or likelihood of being caught if a far more effevtive deterrent, but require effort, focus, and resources by law enforcement.
It's a resource constraint problem and a policy choice. If "they" wanted to set the tone that this type of behavior will not be tolerated, it would require a concerted multi agency surge of investigative and prosecutorial resources. It's been done before, if there's a will there's a way.
Everything I said is true.
>Deterrence is known to have a very limited effect on behaviour.
It is insanely effective when actually enforced. It's not effective when the goal is to make it seem ineffective so that people can evade the system.
>In this case, it's far simpler to prosecute the source.
The "source" is a tool that tomorrow can be in Russia or CHina and you can't prosecute.
And yet criminals still commit crimes. Obviously jail is not the ultimate deterrent you think it is. Nobody commits crimes with the expectation that they'll get caught, and if you only "crucify some people", then most criminals are going to (rightfully) assume that they'll be one of the lucky ones.
I genuinely cannot tell if you are being comically naïve or extremely obtuse here. You need only look at the world around you to see that this does not, and never will, happen.
As another commenter said, this argument is presenting itself as apologia for CSAM and you come across as a defender of the right for a business to create and publish it. I assume you don't actually believe that, but the points you made are compatible.
It is as much the responsibility of a platform for providing the services to create illegal material, and also distributing said illegal material. That it happens to be an AI that generates the imagery is not relevant - X and Grok are still the two services responsible for producing and hosting it. Therefore, the accountability falls on those businesses and its leadership just as much as it does the individual user, because ultimately they are facilitating it.
To compare to other situations: if a paedophile ring is discovered on the dark web, the FBI doesn't just arrest the individuals involved and leave the website open. It takes the entire thing down including those operating it, even if they themselves were simply providing the server and not partaking in the content.
I don't accept this as good faith argumentation nor does HN rules.
For disagreeing on the injection of offtopic hypothetical scenarios as an argument derailing the main topic?
>It should not be possible to create nudes of minors via Grok.
I agree with THIS part, I don't agree with the part where the main blame is on the AI, instead of on the people using it. That's not a bad faith argument, it's just My PoV.
If Grok disappears tomorrow, there will be other AIs from other parts of the world outside of US/EU jurisdiction, that will do the same since the cat is out of the bag and the technical barrier to entry is dropping fast.
Do you keep trying to whack-a-mole the AI tools for this, or the humans actually making and distributing fake nudes of real people?
Both, obviously. For example, you go after drug distributors and drug producers. Both approaches are effective in different ways, I am not sure why you are having such trouble understanding this.
Other AIs have guardrails. If Musk chooses not to implement them, that's his personal irresponsibility.
If the perpetrator is in another country / jurisdiction it is virtually impossible to prosecute let alone sentence.
It is 100% regulatory problem in this case. You just cannot allow this content to be generated and distributed in the public domain by anonymous users. It has nothing to do with free speech but with civility and common understanding of what is morally wrong / right.
Obviously you cannot prevent this in private forums unless it is made illegal which is a completely different problem that requires a very different solution.
The school authorities messed up.
Both are accuntable.
Correction: kids made the pictures. Using Grok as the tool.
If kids were to "git gud" at photoshop and use that to make nudes, would you arrest Adobe?
If kids ask a newspaper vendor for cigarettes and he provides them .. that's a no-no.
If kids ask a newspaper vendor for nudes and he provides them .. that's a no-no.
If kids ask Grok for CSAM and it provides them .. then ?
Meanwhile, the existence/creation CSAM of actual people isn't legal, for anyone no matter the age.
So when Grok provides the illegal pictures then by the same logic it is Grok that is breaking the law.
If parents or school let children roam the internet unsupervised... then?
The explosive sellers that provide explosives to someone without a certification (child or adult) get in trouble (in this part of the world) .. regardless of whether someone gets hurt (although that's an upscale).
If sellers provide ExPo to certified parents and children get access .. that's on the parents.
In that analagy of yours, if grok provided ExPo or CSAM to children .. that's a grok problem,
(Ditto drugs).
It's on the provider to children. ie Grok.
That's actually a good argument. And that's how the UK ending up banning not just guns, but all sorts of swords, machetes and knives, meanwhile the violent crime rates have not dropped.
So maybe dangerous knives are not the problem, but the people using them to kill other people. So then where do we draw the line between lethal weapons and crime correlation. At which cutting/shooting instruments?
Same with software tools, that keep getting more powerful with time lowering the bar to entry for generating nudes of people. Where do we draw the line on which tools are responsible for that instead of the humans using them for it?
The knife (as opposed to sword) example is interesting. In the U.K. you’re not allowed to sell them to children. We recognise that there is individual responsibility at play, and children might not be responsible enough to buy them, given the possible harms. Does this totally solve their use in violent crime? No. But if your alternative is “it’s up to the individuals to be responsible”, well, that clearly doesn’t work, because some people are not responsible. At a certain point, if your job is to reduce harm in the population, you look for where you can have a greater impact than just hoping every individual follows the law, because they clearly don’t. And you try things even if they don’t totally solve the problem.
And indeed, the same problem in software.
As for the violent crime rates in the U.K., I don’t have those stats to hand. But murder is at a 50 year low. And since our post-Dunblane gun laws, we haven’t had any school shootings. Most Britons are happy with that bargain.
The rate of school shootings has dropped from one (before the implementation of recommendations from the Cullen report) to zero (subsequently). Zero in 29 years - success by any measure.
If you choose to look at _other_ types of violent crime, why would banning handguns have any effect?
> Where do we draw the line on which tools are responsible for that instead of the humans using them for it?
You can ban tools which enable bad outcomes without sufficient upside, while also holding the people who use them to account.
No. That is not how AI nowdays works. Kids told the tool what they want and the tool understood and could have refused like all the other models - but instead it delivered. And it only could do so because it was specifically trained for that.
"If kids were to "git gud" at photoshop "
And what is that supposed to mean?
Adobe makes general purpose tools as far as I know.
Anyone skilled at photoshop can do fake nudes as good or even better than AI, including kids (we used it to make fun fakes of teachers in embarrassing situations back in the mid 00s and distribute them via MSN messenger), so then why is only the AI tool the one to blame for what the users do, but not Photoshop if both tools can be used to do the same thing?
People can now 3D print guns at home, or at least parts that when assembled can make a functioning firearm. Are now 3D printer makers to blame if someone gets killed with a 3D printed gun?
Where do we draw the line at tools in terms of effort required, between when the tool bares the responsibility and not just the human using the tool to do illegal things? This is the answer I'm looking for and I don't think there is an easy one, yet people here are too quick to pin blame based on their emotional responses and subjective biases and word views on the matter and the parties involved.
So let's say there are two ways to do something illegal. The first requires skills from the perpetrator, is tricky to regulate, and is generally speaking not a widespread issue in practice. The second way is a no brainer even for young children to use, is easy to regulate, and is becoming a huge issue in practice. Then it makes sense to regulate only the second.
> People can now 3D print guns at home, or at least parts that when assembled can make a functioning firearm. Are now 3D printer makers to blame if someone gets killed with a 3D printed gun?
Tricky question, but a more accurate comparison would be with a company that runs a service to 3D print guns (= generating the image) and shoot with them in the street (= publishing on X) automatically for you and keeps accepting illegal requests while the competitors have no issue blocking them.
> Where do we draw the line at tools in terms of effort required, between when the tool bares the responsibility and not just the human using the tool to do illegal things?
That's also a tricky question, but generally you don't really need to know precisely where to draw the line. It suffices to know that something is definitely on the wrong side of the line, like X here.
> When the sheriff's department looked into the case, they took the opposite actions. They charged two of the boys who'd been accused of sharing explicit images — and not the girl.
Education might be so disrupted you have to change schools.
Besides, getting in trouble for something is already after the fact, the damage has been done. If it can't be done in the first place, or the barrier is too high for most, then the damage would have been prevented.
But this is a recurring dilemma.
We fault and "fine" companies for providing products that harm society all the time
Are you not going to consider the company providing a CSAM machine to be the major one at fault here?
"Boys will be boys", and so on. (https://en.wikipedia.org/wiki/Rape_culture)
The crime is creating a system that lets schoolboys create fake nudes of other minors.
You don't just get to build a CSAM-generator and then be like "well I never intended for it to be used...".
The humans running a company are liable for the product that their company builds, easy as that.
So like Photoshop? Do you want to raid Adobe's HQ?
You should want to protect all of the people in your life from such a thing or nobody.
It is a privately controlled public-facing group chat. Being a chat-medium does not grant you the same rights as being a person. France isn't America.
If a company operates to the detriment and against the values of a nation, e.g. not paying their taxes or littering in the environment, the nation will ask them to change their behavior.
If there is a conspiracy of contempt, at some point things escalate.
Grok is censored, there's no question. Try it yourself. It can't even handle analysing a 19th century woodcut engraving containing topless mythical beings[1]. Grok refused because Grok has guardrails.
[1] https://en.wikipedia.org/wiki/The_Wood_of_the_Self-Murderers...
These raids are entirely political.