"The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand."
Ah.
Right now I'm using abliterated llama 3.1. I have no need for vision but I want to use the saved memory for more context so 3.2 is not so relevant. Llama 3.1 is perfect. But I want to try newer models too.
Until gpt-oss can be uncensored it's no use to me. But if there was nothing erotic in its training data it can't be. And no, I never have it do erotic roleplay. I'm not really interested when there's no real people involved.
Not through the official interface though. Needs to be hosted by a third party. OpenRouter has a generous free tier for both.
I just saw that there is an abliterated version as well. Not sure how to try it though.
And removing the chat interface as much as possible. Many benchmarks are better with text completion models, but they keep insisting on this horrible interface for their models.
Fine tuning is there to ensure you get the output format you want without the extra garbage. I swear they have tuned their models to waste tokens.
Which seems a bit weird, because the customers of the chat interface (ie non-API customers) don't pay per token.
However: I would expect chat interfaces to be charged per query, not per token. End users don't understand tokens, and don't want to have to understand tokens.
If you charge per query, you don't gain anything from extra wordy responses.
It turns out if you generate two LLM responses and ask a judge to choose which is better, many judges have a bias in favour of long answers full of waffle.
The abstract of this paper seems interesting: https://arxiv.org/html/2407.01085v3
> use of [LLMs] as judges [..] reveals a notable bias towards longer responses, undermining the reliability of such evaluations. To better understand such bias, we propose to decompose the preference evaluation metric, specifically the win rate, into two key components: desirability and information mass [..]
(If you're interested, give it a click. I tried to pare this down to avoid quoting a wall of text.)
you kind of need soul for that, and a lot of background knowledge on mythology/fantasy lore, but also tool use to work the world systems.
Technically this worked great, but everything was somewhat bland and generic.
Noteable not-highlights: - there is a shimmer in the nearby forests that keep villagers up at night -> it's an orc camp - there is a mysterious figure in town -> it's a Aragon type ranger
Ask some model to generate a number uniformly between 0 and 100; you get 47 a lot. Or 27. Something like that.
Ask it for a list of numbers, uniformly distributed, and it also often starts with the same number. However later elements in the sequence will converge (imperfectly) to a better approximation of uniformity.
The same for names of the dwarves and paladins in your party.
From time to time I get the event so great and character so compelling, I save the best in Author Note or Lorebook.
The overall atmosphere is effortlessly somewhat Skyrim/Game of Thrones/World of Darkness adjacent.
I'd look for these two models to build this simulator of yours, ie R1 to plan and V3 to fill in the blanks. Oh, or maybe Google Gemini 2.5 or 2.0 to plan the story and DeepSeek V3 to fill in.
Other than that I don't do anything special compared to what other people share on aicg or reddit, etc. Just best practices :) (summaries, good OOC, temperature/top k/top p depending on the model).
Hmmm, maybe spending a lot of time on building interesting, deep, rich persona with deep links to the world (scenario card). 100-300 tokens. There are cards helping in building persona ("Persona Builder" from chub works great). Make it succinct.
I've found myself giggle for hours from the shenanigans of NPCs with my sidekick or just save in the journal best fragments of the gorgeous mental pictures it painted with words.
So at the end the name of the card that works best for me so far: it's called "Loraheim" (also on chub) but from my limited experience good model (DeepSeek V3/R1) + good preset + good persona + passable character will make a great adventure (I've just started a dramatic space opera from the fairly boring "you wake up in the alien dome/zoo").
It can be a great fun, try it :)
Though making it an evil otherwordly creature is a bit extreme, it's at least similar to what a flexible GM can do. In my DMing days, I would often develop new paths that integrated into the whole inspired by things my players noticed/suspected.
You are right though and it's not that I completely dislike the LLMs "flexibilty" and openness to suggestions. However, it's also super easy to use it for "cheating". E.g. it generated a scenario with an evil entity about to attack me and some friendly NPC and I could "solve" that problem by telling the NPC "remember the device I gave you last week and told you to always keep on hand? pull the trigger now!" (that never happend, at least to the LLMs knowledge) and the LLM made up some device that shot a beam of magic light at the creature and stopped it.
It's a well-understood self-contained use-case without many externalities and simple business models.
What more, with porn, the medium is the product probably more than the content. Having it on home-media in the 80s was the selling point. Getting it over the 1-900 phone lines or accessing it over the internet ... these were arguably the actual product. It might have been a driver of early smart phone adoption as well. Adult content is about an 80% consumption on handheld devices while the internet writ large is about 60%.
Private tunable multi-media interaction on-demand is the product here.
Also it's a unique offer. Role playing prohibited sexual acts can be done arguably victim free.
There's a good fiction story there... "I thought I was talking to AI"
So I started with scraping and cross-reference, foaf, doing analysis. People's preferences are ... really complex.
Without getting too lewd, let's say there's about 30-80 categories with non-marginal demand depending on how you want to slice it and some of them can stack so you get a combinatoric.
In early user testing people wanted the niche and found the adventurous (of their particular kind) to be more compelling. And that was the unpredictable part. The majoritarian categories didn't have stickiness.
Nor did these niches have high correlation. Someone could be into say, specific topic A (let's say feet), and correlating that with topic B (let's say leather) was a dice roll. The probabilities were almost universally < 10% unless you went into majoritarian categories (eg. fit people in their 20s).
People want adventure on a reservation with a very well defined perimeter - one that is hard to map and different for every person.
So the value-add proposition went away since it's now just a collection of niche sites again.
Also, these days people have Reddit accounts reserved for porn where they do exactly this. So it was built after all.
[1] https://aella.substack.com/p/fetish-tabooness-and-popularity...
If people were polled what they want to see on social media, few would say things that are inflammatory, upsetting, divisive, etc but those as we know are strong drivers of engagement.
It's because you're polling for affinity or disclosed preference not for the actual engagement drivers.
For instance, if a male says they watch male pornography, they are labeling, or at least stating an affinity to a sexual identity.
However, the identities people choose to own are not the same as the preferences they actually have.
Instead if you track things like scroll velocity, linger time, revisitation, the time distance (such as 2 days apart instead of 5 minutes) a different story emerges.
For instance a given male could frequently look at male pornography but for all kinds of social reasons not want that affinity so they'd never even internally ideate the preference although their behavior of frequenting male content will be there regardless.
That's one of the problems with this approach is that not many people want to own all the social identities which map to their preferences so they don't openly identify it.
There (maybe) three levels of acceptance: admitting it to oneself, to others, identifying with it. And honestly these have a poor mapping to actual engagement with explicit content. You can have a (insert sexual affinity) rights activist who does not look at explicit content and someone protesting them who does all the time.
That's because those are two entirely different things. If you polled people and asked them "what causes you to spend more time on social media", then at least some self-aware folks would likely identify conflict, "someone is wrong on the Internet" (https://xkcd.com/386/), etc. That doesn't mean that's "what they want to see on social media", that means that's "what gets them to spend more time on social media".
Didn't reddit remove porn?
Neither of them intended to be porn sites. It's kind of a natural occurrence on UGC sites . Look at Civitai...
Credit card processors are kinda weary of it for some legal reasons I'm not qualified to enough to really understand.
For moralizing activist reasons. It's nothing to do with legality. With any luck eventually they'll inadvertently trample a sacred cow of whichever party is currently in power and we'll finally get sane legislation outlawing their overbearing nonsense.
Anyway "child porn" as well as the broader "legal reasons" fails to explain the US payment processors' moves to block all sorts of content and products over the years. Even including porn that isn't user uploaded (and thus has proper records keeping).
Anyway you've circled back to user generated content. But again, that's far from the only thing that payment processors have discriminated against over the past couple of decades.
Here's the most notable case I'm aware of: https://en.wikipedia.org/wiki/GirlsDoPorn
Then there was the recent drama with civitai https://civitai.com/articles/14945/credit-card-payments-paus...
I believe some of these sex trafficking laws implicated a broad sweep in their litigation and Visa/Mastercard doesn't want to have to go to court over a these things.
snort
let's say you publish a Steam game how to be a school shooter and shoot kids, wouldn't that lead to real school shootings ?
who can definitely say that computer generated content about criminal behavior, won't lead to real crime with real victims?
Let's be specific: Rape, incest, necrophilia, bestiality, and pedophila ideation.
I think we can all agree (1) these are harmful, anti-social behaviors that we do not want in our society, (2) people don't choose to have these desires, (3) most people who have them have no desire to actually traumatize others, (4) people who have these struggle with it.
These multi-media AI role-play environments would allow that type of engagement without any harm.
Now given all this, I am not a psychologist and do not know if that's part of how someone unfortunate enough to have those inclinations can deal with it healthily.
But if it is, now it exists and hopefully we can see less of it in the real world. I'm all for harm reduction if this is a way to get there.
I am at a principled level uneasy with what’s fundamentally a sort of prior restraint (you haven’t yet hurt anyone but this may increase the likelihood and/or be an effective proxy to lock up those who are more likely to do so) but also see a really strong case for doing it given the fact that these are arguably the most antisocial behaviors one can imagine.
Typing a prompt in an AI box to make art has fewer real-world victims than performing the acts, filming them, and then sharing the videos.
I think that's inarguable. Maybe it's still unadvisable and someone should be in talk therapy. I have no idea. But at least nobody is actually getting molested and retraumatized in the ai art scenario.
If someone is spending their time using comfyUI drawing pictures instead of stalking the local middleschool, I'd hesitate to say mission accomplished ... but maybe I should?
People's time is finite. They can't be doing both. If the real is substituted for the imaginary then the real can no longer happen because that time is spent.
This all falls flat on me though. It's like showing me the lewdest most shocking story and then saber rattle about keyboards and word processors.
I'm fully aware of the wild things people do with drawing programs.
We're all adults here. It's fine.
Because both possibilities are plausible, it’s hard to know which is correct.
I'd really defer to experts.
I try to make tools in good faith and hope they're used responsibly to make the world a better place.
I'm not a clinical psychologist nor can I pretend to understand medical literature like someone with a PhD
It doesn't mean I can do things recklessly. Instead it's an acknowledgment of when I need to defer to somebody else just like I need to call up an attorney for legal stuff or an accountant for tax stuff.
A: I’m curious about X.
B: We should trust the experts!
Sure, but what do the experts say? That was my entire question.
Anyhoo, The current "state of the art", as-scientific-as-we-currently have it findings are that for pedophilia, consuming content unfortunately normalizes and drives increasing urges, instead of giving them same outlet. It's a very very tricky area because current thought is also that pedophilia is "not curable" - it is sexual orientation thay we as society find unacceptable (me included, fwiw), so... Repression, wildly and rightly disawowed for other sexual orientations, is the current direction for pedophilia - I.e. Current thinking is that "victim-free" pornography consumption nevertheless tremendously increases actual risk to actual kids in the vicinity of the consumer.
Until relatively recently I was the technologist on a highty horse about online freedoms and largely still very much am. But in this specific area I've also had some semi personal experiences with pedophiles and my level of empathy to them has dropped to near zero and my level of empathy toward their victims has gone even further through the roof. Sometimes in technologist circles we think of this as edge case not worth consideration, but reality, very very unfortunately, is much much darker.
Don't get me wrong : I'm pro pornography freedoms, think it'd be huge fun to have a sexy high quality chatbot, and I find vast majority of those railing against it to be hypocrites with dishonest ulterior motives - and don't get me started on all the tangential "for the children" crap that religious rights tries to enact, as opposed to actually help children and families ;-<
But to the question of "is harmless pornography Indulgence better for paedophile and society", current thinking is "very much no".
most of the perpetrators of child sexual abuse are victim's close people: teacher/cousin/brother/uncle/father/etc.
the reinforcement of their lust will only remove whatever remaining barrier against such repulsive behavior, and once the novelty from synthetic CP wears out, it will create urge to commit real crime with real victims
You can’t really research when the only thing you can be certain of is the known real cases. It’s much harder to quantify people that only have it in their head.
It's not completely clear if it's just a spurious correlation or if there's a real causation, but eh, more training data + neutral alignment training is how humans train AIs, I don't see why would some says that's not how baby humans are to be trained.
Ie yes it’s bad and in an ideal world nobody would do it. I see trying to restrict or ban it as the greater of two evils.
> who can definitely say that computer generated content about criminal behavior, won't lead to real crime with real victims?
I can’t tell if you’re being sarcastic but there has been no found link between violent video games to violent crimes, despite it being researched extensively:
https://www.apa.org/news/press/releases/2020/03/violent-vide...
https://link.springer.com/article/10.1007/s10964-019-01069-0
https://pmc.ncbi.nlm.nih.gov/articles/PMC6756088/
https://elifesciences.org/articles/84951
Of course, that hasn’t stopped video games being blamed for violence by the “think of the children” crowd and certain politicians:
https://en.m.wikipedia.org/wiki/Family_Entertainment_Protect...
https://www.theatlantic.com/technology/archive/2019/08/video...
Especially when shootings occur by white perpetrators:
https://www.apa.org/news/press/releases/2019/09/video-games-...
The same narrative plays out for porn, despite the research findings being the same:
https://www.utsa.edu/today/2020/08/story/pornography-sex-cri...
But blaming violent video games or pornography is an easy scapegoat
And to this day, military recruiters use the AC130 mission in CoD to convince people to become aerial gunners.
Feels like you're falling into the same trap that Senator Lieberman did in the 90s, and just another spiritual successor to Satanic panic.
You might find this part funny. At first, I thought they had automated their coding for SAP or other ERP databases. Then, they started talking about how realistic the body parts were. I paused staring at the screen. The sad reality clicked.
{
"race": "elf",
"horny": false
^^^^^^^^^^^^^^
Unsupported value.Maybe nothing wrong with that, but it might mean that the perceived weaknesses don't generalize to an area of the model that hasn't been lobotomized.
* using safety the way OpenAI have been using the term, not looking to debate the utility of that.
Meanwhile, the anti-porn side has a formidable alliance:
Right-wing, religiously-motivated anti-porn activists. Left-wing, feminism-motivated anti-porn activists. Big corporate types with lots of $$$$ to spend who want their customer support chatbot to be completely SFW at all times. AI safety folk who think keeping the model on a tight leash is an ethical obligation, lest future iterations take over the world. AI vendors who are keen on the yes-it-might-take-over-the-world narrative. AI vendors who just don't want their developers having to handle NSFW stuff in work. Politicians who don't know a transformer from a diffusion model, but who've heard a chorus of worries about lost jobs and AI bias and deepfakes and revenge porn.
These people will speak up in public at the drop of a hat.
erotic roleplay, imo, is much less harmful than using LLMs as surrogate partners. porn and sex workers have existed for millenia. they're an outlet for sexual tension. they don't alleviate feeling lonely or provide an alternative to human companionship.
I'm worried we'll produce a generation of hikkikomoris, who eschew human connection for sycophantic machines that always listen and never breaks their heart.
But if that person is applying AI safety techniques like concept erasure to remove the model's ability to output porn, is that not anti-porn in the most literal sense?
I am playing around with interactive workflow where the model suggests what can be wrong with a particular chunk of code, then the user selects one of the options, and the model immediately implements the fix.
Biggest problem? Total Wild West in terms of what the models try to suggest. Some models suggest short sentences, others spew out huge chunks at a time. GPT-OSS really likes using tables everywhere. Llama occasionally gets stuck in the loop of "memcpy() could be not what it seems and work differently than expected" followed by a handful of similar suggestions for other well-known library functions.
I mostly got it to work with some creative prompt engineering and cross-validation, but having a model fine-tuned for giving reasonable suggestions that are easy to understand from a quick glance, would be way better.
For example: make the suggestion output an object with multiple fields, naming one of them `concise_suggestion`. And make sure to take advantage of the `description` field.
For people not already using structured output, both OpenAI and Anthropic consoles have a pretty good JSON schema generator (give prompt, get schema). I'd suggest using one of those as a starting point.
But then people used it for erotic RP, and it became a PR disaster, and the author blamed his pervert customers. Never mind that certain characters who turned up a lot in AI dungeon stories turned out to be from a fantasy writing site the author had used for fine-tuning material (without permission of course) and he hadn't filtered OUT the dirty stories to put it like that.
i’m an adult.
Perhaps you could elucidate further on this subject? I'm mostly into books from 1800-1985 or so and don't know much about contemporary literary fashion.
Edit: Jean M Auel was extremely common in occidental households a few decades ago, especially the first and second books about Ayla, I'd wager much, much more common than cyberpunk.
Same goes for books by Alex Comfort.
LLMs are also hilariously bad at what makes erotica hot in the first place.
They provide usage rankings [1] and the top 10 applications, in terms of tokens used, are:
1. AI coding agent 2. AI coding agent 3. AI coding agent 4. Library for calling LLMs 5. Role play chat 6. Role play chat 7. Role play chat 8. General purpose chat 9. AI coding agent 10. Role play chat
Certainly, the coding agents are burning through far more tokens. No doubt about that.
And there's undoubtedly a major bias introduced by the fact chatgpt and claude $20/month accounts are heavily discounted, so if your use is SFW why pay more to get an uncensored model?
But overall, to me, the evidence seems pretty robust.