There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0].
All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product.
This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this.
It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all.
[0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else.
So I don't see how they're "damned if they don't".
The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image.
That's what they want to avoid and that's why they are trying to take preventive measures.
I'd be very worried about this in their shoes and acting very similarly.
"They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!"
Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again.
"They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
Wow. You act as if that's something new :D
There's even an actual job for doing just that: it's called an editor. You know, the people who work in media, publishing houses, TV, radio, papers, magazines, that kind of thing. It's what they do. Everywhere. Editing content. Removing comments, changing words, making sure the views represented in the publication match the intentions and policies of the investors and parent organisations, etc.
You act as if something is new and evil just because it's done with computers. It isn't. It has been the case for as long as papyrus and cave paintings have been around. Back then it was don't anger the chief/king/Pharao, today it's don't anger the Twitter mob or the hand that feeds you. Same difference.
Games are a medium just like magazines or online blogs and comparing blocky dongs in Minecraft with explicit child pornography says a lot about your understanding of "creative expression". Really makes me wonder sometimes.
When comparing the Minecraft scenario to the AI Dungeon scenario under the purview of "doing creative things with games," I don't find the two scenarios to be much different. However, for one of those things, it's a possibility within the vast play space of Minecraft - you could even create genitalia and display profanity with just alphabet blocks - but for the other scenario, depending on how you see it, it's content that is generated, served, and potentially stored by AI Dungeon. I'd be concerned about the publicity if I was AI Dungeon as well.
This is not a technical consideration, it is just PC at its worst.
You either deny that games are a medium or you're under the impression that whatever you read in news publications, books, see in films and TV is the result of the unfiltered creative outlet of the producers. News flash: it isn't.
There's heavy editing going on everywhere and just because it targets a specific computer model in this case instead of a more "traditional" product like a novel, film or stage play doesn't make an iota of difference.
In fact this demonstrates the maturity of the technology and its use if it gets the same treatment as every other public media.
Which forum products do you enjoy and which do you not?
Centralised systems is the only reason why you get these problems.
the Internet's original sin!
the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
"Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional."
As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists.
Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition
A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.
The distinction between "thoughts" and "content" here could be made as "whether it is shared with others".
For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that?
What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer?
In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature.
If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing".
What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic.
It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
>"corporate self harm for the sake of wokeness"
"Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.