> [...]
> Sexually explicit content
Dear tech companies. Sexually explicit content is not harmful. Why are you all run by puritans? I don't even want to make edgy porn, I just want to be treated like an adult.
Sure, there are always jailbreaks, but then the narrative changes from "we made a model that tells erotic stories to children" to "this ingenious teenager figured out a way to hack our model to make it produce erotic stories." In other words, jailbreak move the fault from the model producer to the model user.
It's also worth keeping in mind that erotica comprises a surprisingly large portion of fiction easily available on the internet for free, and "unfiltered" models tend to produce that kind of content unprompted (see e.g. the original Mistral). The major AI labs are probably filtering it out, but I suspect they can't go too far there, as having a model that is good at fiction is something they actually want.
Then there are the non-chat-gpt-app use cases (like customer support chatbots, automatic summarization etc), for which unprompted erotica is highly inappropriate. Those are the "business travelers" of AI, not the first thing one thinks of when talking about who uses AI models, but extremely important nonetheless.
It's hard to think of a scenario where there's a child technical enough to run Gemma 3 locally but somehow unable to access any other written erotica. Project Gutenberg is full of erotic textual content and I haven't heard of anyone calling for that to be banned.
>Then there are the non-chat-gpt-app use cases (like customer support chatbots, automatic summarization etc), for which unprompted erotica is highly inappropriate. Those are the "business travelers" of AI, not the first thing one thinks of when talking about who uses AI models, but extremely important nonetheless.
And how many of these are going to be using Gemma, when Gemini over the API is cheaper, faster and easier to use?
Companies and government organizations who have sensitive data are still unwilling to use these models over any API they don't host themselves.
I work in this space in the EU, and this is absolutely a problem.
The reason you're struggling to understand is that you're thinking about this logically.
Adult content is obviously freely available to any child or adult with minimum technical skills. What makes LLMs different is that it's "the new thing" and people respond differently to "the new thing".
Then it's up to users (or parents, in the case of children) to choose the adequate version for each purpose. Just like there are child-friendly movies and adult-only movies, and no one beyond fringe puritan crusaders would say that the latter should outright not exist.
Well here you still have the same problem, since they're not gonna release an actually uncensored version, that tells you how to do awful things (or indeed, that tells you to do them).
So then you'd have censored and less censored, and it would still be a matter of where to draw those lines.
What I mean is a model for all audiences and an adult model, and the line would be drawn at the law of the country producing it (if it's something that would be legal to publish for a human author at a website, then it should be allowed as an LLM response). So erotica would be fine, while instructions for making a bomb wouldn't.
Since this is just giving the model directly, there's no ability to do any filtering as part of inference, so I would imagine you have to assume the worst (intent) on any input coming into it.
It’ll get easier once the costs of building foundational models go down and human labeling gets automated. Sit tight, models that’d be creative and amazing at generating erotic content are certainly coming.
"The Most Intolerant Wins: The Dictatorship of the Small Minority"
https://medium.com/incerto/the-most-intolerant-wins-the-dict...
People like pornography. They'll as soon ban alcohol again (which worked so well last time)
Alcohol is another good example.
I'm sure the hysterical puritans of the past will come out any day now and admit that they weren't even 1% correct in their assertions.
My understanding is that this is one of their complaints
"I have a right to live in a society that perfectly adheres to my personal morals" is not how companies or people should operate in a pluralistic society, despite Nassim Taleb's claim that the intolerant minority wins.[0]
[0] https://medium.com/incerto/the-most-intolerant-wins-the-dict...
Last time only some groups of enthusiasts were willing to work through bugs to even run the buggy release of Gemma
Surely nobody runs this in production
The risk of the model generating illegal content and then the company getting bad PR from vultures in journalism simply outweighs any benefits of including this content in the training data.
This is also why you will never see the big companies release a capable open weight image or video gen model.
This is completely unsubstantiated. The original Sydney (Bing AI) was violently unhinged and this only drew more users; I haven't met a single person who prefers the new Bing AI to the old Sydney, and for that matter I haven't even heard of anyone using Bing AI for ages now they toned it down. Trust in journalists is at an all-time low ( https://news.gallup.com/poll/651977/americans-trust-media-re... ) and America recently elected an extremely unorthodox president in big part due to the sheer hatred of the media shared by a large proportion of the population. Even the most hardcore social conservatives aren't calling for companies to censor the training of open source models so they don't produce adult textual content even when prompted to do so; it's not a political issue.
See this: https://github.com/orgs/community/discussions/72603
Who is their target group for small local models that benchmark inferiorly to their proprietary solution (Gemini 2.0) then, if not hobbyists and researchers?
Every model I've tried so far is bad at distinguishing sexually explicit content from mere nudity, and many models are bad at distinguishing nude from non-nude. I don't know about Gemma 3 but Google's large commercial Gemini models refuse (or formerly refused; haven't tried recently) to tell me anything useful about images containing human figures. I assume that this is due to aggressive "safety" measures. On a technical basis, I assume that a model that can distinguish 10 different breeds of dog should also be able to usefully describe images of people wearing swimsuits, nude people, and people engaged in sexual intercourse.
If this gives me the "aschually as a ethical safe harmless assistant I can't ..." spiel on anything mildly mature, that would be very disappointing. I'll run a test with Berserk and see how it goes.
I'm not a big believer in abliteration, it seems to always hurt performance. Safety should be handled by a separate system, no need to cripple the actual LLM.
You'll want to use custom models to segment the manga (panels, speech bubbles), OCR the text, translate (gemma punches above it's weights for this part).
That said, I've been experimenting with using Pixtral to do the analysis part with okay-ish results (providing individual panels with the character names) but it'll still mix up the characters when they're drawn differently.
> I'm not a big believer in abliteration, it seems to always hurt performance.
Agreed, it's fun to play with but it increases halucinations. And for creative writing, it makes the model write more compliant characters (they'll give in too easily during negotiations, rather than refuse, etc)
Could probably be improved with more targeted abliteration.
Oh, there are loads of porn enjoyers working in such companies - but traditional professionalism means you leave the porn at home during the work day. It is, after all, NSFW.
So at the meeting where censorship decisions were being made, even a weak argument for censoring explicit content will be accepted unopposed.
The Gemma family is a family of local models!
Or perhaps the measurement of improvement was biased. If a model doesn't understand the word gay there would certainly be people who would find real world use of the model to be substandard.
Did the assessment of what counts as improvement come from the same community that decided that excluding things with 'gay' was cleaning the data?
Actually, it will happen naturally and eventually. Just look at Apple Vision Pro which still don't have VRChat support, and compare how deeply DOA it has been to other VR headsets that are clearly nowhere near as important. Or "Metaverse" that were all explicitly SFW.
This effect can even be seen in the Apple App Store itself. Who uses App Store? You flow into App Store through porn-enabled platforms, such as web or social media. No one browses App Store as a content. What does it not have? Pornography.
lawyers.
(on both sides)
nipple BAD.
exploding someone into bits GOOD.
Early models were censored, making uncensored releases have bad optics.
If the first models had been uncensored, no one would care if another was added.
It’s unsafe for that reason, so you absolutely needed both censored and uncensored. It wasn’t an accident.
A sexualized fine-tune yes, but that's because you have to make them overly horny to overcome the original censorship.
Nothing prevent them to train a model that will have an appropriate level of sexual content (that is, only upon user explicit request) the same way they train it not to have sexual content at all.
The reason they do that is because they are American companies, the same companies who also censored nude paintings and statues from European museums' pages.
This is just silly because it only takes one AI company to defect and start enabling it, and the problem is already pretty bad even without AI.
I think all of the solutions are demand-side, not supply side. I would expect differential reproductive rate trends between populations with and without proscriptions on ersatz reality consumption (i.e. aniconist Muslims, Mennonites, etc.) to accelerate