Hacker News Reader

Comment by etchalon | Hacker News Reader

[flagged]

if a user uses a tool to break the law it's on the person who broke the law not the people who made the tool. knife manufacturers aren't to blame if someone gets stabbed right?

This seems different. With a knife the stabbing is done by the human. That would be akin to a paintbrush or camera or something being used to create CSAM.

Here you have a model that is actually creating the CSAM.

It seems more similar to a robot that is told to go kill someone and does so. Sure, someone told the robot to do something, but the creators of the robot really should have to put some safeguards to prevent it.

one time is perhaps excusable as the robot's creators didn't know that would happen. If it happens and the creator then advertises the robot can kill people, of course he's now a criminal

If the knife manufacturer willingly broke the law in order to sell it, then yes.

If the manufacturer advertised that the knife is not just for cooking but also stabbing people, then yes.

if the knife was designed to evade detection, then yes.

Text on the internet and all of that, but you should have added the "/s" to the end so people didn't think you were promoting this line of logic seriously.

If a knife manufacturer constructs an apparatus wherein someone can simply write "stab this child" on a whim to watch a knife stab a child, that manufacturer would in fact discover they are in legal peril to some extent.

I mean, no one's ever made a tool who's scope is "making literally anything you want," including, apparently CSAM. So we're in a bit of uncharted waters, really. Like mostly, no I would agree, it's a bad idea to hold the makers of a tool responsible for how it's used. And, this is an especially egregious offense on the part of said tool-maker.

Like how I see this is:

* If you can't restrict people from making kiddie porn with Grok, then it stands to reason at the very least, access to Grok needs to be strictly controlled.

* If you can restrict that, why wasn't that done? It can't be completely omitted from this conversation that Grok is, pretty famously, the "unrestrained" AI, which in most respects means it swears more, quotes and uses highly dubious sources of information that are friendly to Musk's personal politics, and occasionally spouts white nationalist rhetoric. So as part of their quest to "unwoke" Grok did they also make it able to generate this shit too?

This is really amusing to watch, because everything that Grok is accused of is something which you can also trigger in currently available open-weight models (if you know what you're doing).

There's nothing special about Grok in this regard. It wasn't trained to be a MechaHitler, nor to generate CSAM. It's just relatively uncensored[1] compared to the competition, which means it can be easily manipulated to do what the users tell it to, and that is biting Musk in the ass here.

And just to be clear, since apparently people love to jump to conclusions - I'm not excusing what is happening. I'm just pointing out the fact that the only special thing about Grok is that it's both relatively uncensored and easily available to a mainstream audience.

[1] -- see the Uncensored General Intelligence leaderboard where Grok is currently #1: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard

> everything that Grok is accused of is something which you can also trigger in currently available open-weight models (if you know what you're doing)

Well, yes. You can make child pornography with any video-editing software. How is this exoneration?

I'm not talking about video editing software; that's a different class of software. I'm talking about other generative AI models, which you can download today onto your computer, and have it do the same thing as Grok does.

> How is this exoneration?

I don't know; you tell me where I said it was? I'm just stating a fact that Grok isn't unique here, and if you want to ban Grok because of it then you need to also ban open weight models which can do exactly the same thing.

> that's a different class of software. I'm talking about other generative AI models

And the article is talking about a social media site. A different class of software and company.

> if you want to ban Grok

Straw man. Nobody has suggested this.

I think he is talking about France who does very much seem like they want to ban X and Grok?

If Grok is a child pornography tool then it will be banned. If Elon doesn't want Grok to be banned, he can disable the child pornography feature.

Agreed, thankfully, there is no CSAM feature, and they patched it in a couple of days so that it would stop obeying requests to do so and banned the users abusing Grok for that purpose. Once they prove they have put the appropriate mechanisms in place to make sure it doesn't happen again it's fine and dandy, no?

> Agreed, thankfully, there is no CSAM feature, and they patched it in a couple of days so that it would stop obeying requests to do so and banned the users abusing Grok for that purpose.

The order, over several days, was:

* banned users and took down tweets requesting the content, without taking down the content that they clearly knew of, since they responded against the requests that generated it.

* Made the feature paid-only.

* Took down content and restricted the functionality.

> Once they prove they have put the appropriate mechanisms in place to make sure it doesn't happen again it's fine and dandy, no?

I’m not sure all jurisdictions involved will see knowing generation and dissemination of nonconsensual, including child, pornography as the kind of thing where “I promise not to do itt again, so we’re cool, right?” works, but we’ll see.

If by patched you mean he made it cost money.

Can you go on Twitter right now and generate CSAM using Grok?

> France who does very much seem like they want to ban X and Grok?

Source? I’m not seeing that in the French-language press.

Well you could not sue the video-editing software for someone making child pornography with it. You would, quite sanely, go after the pedophiles themselves.

We don't go after Adobe for doing that. We go after the person who did it.

Maybe tying together an uncensored AI model and a social network just isn't something that's ethical / should be legal to do.

There are many things where each is legal/ethical to provide, and where combining them might make business sense, but where we, as a society have decided to not allow combining them.

You cannot build a CSAM generator, period. CSAM means Child Sexual Abuse Material -- material created through the sexual abuse of children. If it came out of a generator, it is not, by definition, Child Sexual Abuse Material.

CSAM is the woke word for child pornography, and it's a bad word, like how pedophiles should be called pedophiles and not MAPs. The original term, child pornography, is better.

Every AI system is capable of generating CSAM and deep fakes if requested by a savvy user. The only thing this proves is that you can't upset the French government or they'll go on a fishing expedition through your office praying to find evidence of a crime.

>Every AI system is capable of generating CSAM and deep fakes if requested by a savvy user.

There is no way this is true, especially if the system is PaaS only. Additionally, the system should have a way to tell if someone is attempting to bypass their safety measures and act accordingly.

> if requested by a savvy user

Grok brought that thought all the way to "... so let's not even try to prevent it."

The point is to show just how aware X were of the issue, and that they chose to repeatedly do nothing against Grok being used to create CSAM and probably other problematic and illegal imagery.

I can't really doubt they'll find plenty of evidence during discovery, it doesn't have to be physical things. The raid stops office activity immediately, and marks the point in time after which they can be accused of destroying evidence if they erase relevant information to hide internal comms.

Grok does try to prevent it. They even publicly publish their safety prompt. It clearly shows they have disallowed the system from assisting with queries that create child sexual abuse material.

The fact that users have found ways to hack around this is not evidence of X committing a crime.

https://github.com/xai-org/grok-prompts/blob/main/grok_4_saf...

Is there evidence this is the real prompt?

Grok makes it especially easy to do so.

What makes Grok special compared to random "AI gf generator 9001" which is hosted specifically with the intent of generating NSFW content?

If AI GF Generator 9001 is producing unwilling deepfake pornography of real people, especially if of children, feel free to raid their offices as well.

> What makes Grok special

X. xAI isn’t being raided. X is. If Instagram bought a girlfriend generator and built it into its app, it would face liability as well.

>Every AI system is capable of generating CSAM and deep fakes if requested by a savvy user. The only thing this proves is that you can't upset the French government or they'll go on a fishing expedition through your office praying to find evidence of a crime.

If every AI system can do this, and every AI system in incapable of preventing it, then I guess every AI system should be banned until they can figure it out.

Every banking app on the planet "is capable" of letting a complete stranger go into your account and transfer all your money to their account. Did we force banks to put restrictions in place to prevent that from happening, or did we throw our arms up and say: oh well the French Government just wants to pick on banks?

Every artist is capable of drawing CSAM. Every 3D modeler can render CSAM. Ban all computers !!

Well every human can be an artist with some training. I guess the solution is to ban humans.

And the artist is punished for doing so. Thank you for proving my point.

But we don't ban pencils do we?

right and X isn't having it's GPUs banned

You can use photoshop to create CSAM too, should that be banned?

Reply on news.ycombinator.com