AI Dungeon will block certain words, review content flagged as inappropriate
latitude.io
latitude.io
The key reason is perhaps this, buried deep in the text: "We have also received feedback from OpenAI, which asked us to implement changes". Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it's easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors.
Whatever may be said of the robocensor they've thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they've handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about "misinformation" and throwing shade onto others, is starting to look like a pattern.
distance = ((x1-x0)**2 + (y1-y0)**2)**.5
https://developer.mozilla.org/en-US/docs/web/javascript/refe...
I have a BS in math, so obviously I'm familiar with the math notation, but I code like an English major.
This was by a person that sprinkled lazy backfilling behavior throughout the codebase, created magic method_missing dispatches against polymorphic ActiveRecord relationships, and exposed the database as a service through a RESTful DSL that allowed arbitrary method dispatch.
I'm glad I spend most of my time in statically typed, math and binary-friendly codebases now.
Frankly, I've always assumed the devs have access to my sessions. Whether it's AI Dungeon themselves or OpenAI, you know that data is being harvested. And at this juncture in text AI development, that kinda makes sense. Obviously I would prefer privacy where possible, but these companies are data hungry and they own the park we're playing in. So it only seems fair.
We'll have to wait awhile for GPT-3 like models to democratize before we can expect real privacy. In the meantime, just err on the side of assuming all input to OpenAI systems is being harvested.
> and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific
Funnily enough, I didn't run into any issues with any of my NSFW sessions since they implemented the filters. So I guess the problems are with SFW sessions so far :P
Anyway, I've got to say I'm kinda happy about AI Dungeon tackling this problem. They made it clear in their announcement that they aren't targeting NSFW content in general, just the one subject. The AI has a tendency of shoving that subject randomly into sessions, which isn't great. If they can eventually filter that filth out without affecting quality otherwise, I think the service will be better for it.
What a time to be alive!
This kind of thing happens everywhere, everyday in TV stations, editorial offices, at publishing companies, radio stations - all kind of media really.
Depending on the political or moral views of the parent organisation or investors, this content censoring/massaging is everyday business and shouldn't shock or surprise you in the slightest.
All of this just smacks of the old, worn-out argument that "it's different because it uses computers!"
Editor is a literal career field, and has been for years and years.
Surely the important distinction is not the text itself but which character a reader empathises with — the monster or the victim.
(Personally I don’t understand why violent horror as a genre exists, and literally cannot empathise with people who enjoy it. Nonetheless I recognise that enjoyment of horror does not make one a monster).
There's a bunch of other stuff about The Nightmare of Consciousness and so on in his book The Conspiracy Against the Human Race.
What's wrong with it as entertainment?
"""You’re entertained by torture and murder of characters who had done nothing deserving of such suffering."""
That part of me wants to continue with a long rant, but by this point my rational self can sit my emotional self down with a nice cup of tea and a biscuit.
Golf stimulates some brains and understimulates others.
Sympathy, humility against my own feelings, sure. Not empathy.
Not sure how much of a line that is, didn't "It' have an underage sex scene in it? Wouldn't it get banned here, too?
> AI Dungeon will continue to support other NSFW content, including consensual adult content, violence, and profanity.
Anything sexual? Oh no no no! The children!
Americans are fucking weird...
I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this.
Our character had a AI man approach the door of their house with magic "love potion" berries. We tried to get our character to not eat the berries, but the AI "tricked us" into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character.
We closed the laptop and haven't brought this up again.
It's certainly the impression I got from watching some youtubers playing it before and after the monetization change.
Also, I don't agree with the example: Steve wouldn't be invited back after either act. YMMV.
Rape, on other hand, could hardly serve any acceptable in-game purpose (or real life purpose for that matter). Its purpose is to terrorize and psychologically maim -- something you never need to do to an NPC. An ally who rapes someone in-game is not furthering the quest, and moreover is doing something that is well outside the bounds of typical PC behavior.
Source?
And a murder-victim-to-be still probably wouldn't be triggered, since s/he hasn't "experienced" it yet.
That's how I interpreted the comment anyway :)
...then you probably won't succeed anyway, as triggers tend to be random associations.
But then again, what about the second-order effects on friends and family as a result of either sexual abuse or violence? Maybe the trauma belonging to the survivors themselves simply overpowers the rest (or not). What about people who survive murder attempts? Maybe being taken advantage of and treated as powerless applies more to sexual than physical trauma, since there are many cases where physical violence is the result of both sides retaliating in equal measure, or sometimes honorably, like for sport. I'm not sure.
Most tabletop roleplaying games have mechanics about killing things but no mechanics about sexual violence, so that tends to set expectations too.
I think you expect fighting but rarely sex in a game.
Sometimes players justify their actions with "but that's what my character would do"; there is a popular rpg.stackexchange post about it: https://rpg.stackexchange.com/questions/37103/what-is-my-guy... .
Like, imagine that you've stumbled on a weird internet story where in the first page someone is approached with magic "love potion" berries but refuses to eat them. That is a solid indicator of what genre the story is. If you had to bet lots of money, what's the probability that the second page will contain something horrific versus the probability that the "seduction" just fizzles out and becomes irrelevant? If you see a movie where the first scene involves a creepy character making a pass, wouldn't you be fairly certain that an escalation of that will follow later? It's like Chekov's gun, once it's there, it almost certainly means that the story is about that - perhaps it could be turned into a "just revenge" story by inserting descriptions of some heroic rescuer or references to how the protagonist expected this to happen in order to punish the assaulter, because stories like that have been written, but a "mediocre" outcome where eventually nothing dramatic happens and the protagonist just gets out won't be generated, because that doesn't get written about, the training data says that such a result is very unlikely. It's obviously a problem, but since it's a "honest probability" based on tropes we see in actual literature, it's going to be hard to fix; the system expects escalation and drama (because all the training stories had that), so you can choose the direction of that escalation, but it won't allow you to have a "non-story" where the suggested drama results in nothing dramatic.
(And the predictive processing theory of cognition, and how that's surface-level-related to the original topic of GPT-3...)
Even in a "you are flying trough space, there is a radio signal coming from a planet" setting there is no way to just ignore the signal and keep flying: The AI decided that signal is the plot, and you gonna investigate it whether you want to or not.
Opening a time machine portal from whatever medieval kingdom I was in to teleport to San Francisco and going to the Open AI office, running into Eliezer Yudkowsky and interviewing Sam Altman about Open AI, etc.
It was pretty easy to shift gears - you could force actions.
"Ask the receptionist if Sam is in"
"She says he is not"
I input: "Sam comes out of his office and walks down the hall"
"Look at the receptionist and say, he's right there."
"She stares at you blankly"
You could input story and then use the story lines that you had written in to advance things.
I eventually got tired with it because it was too free form so there wasn't much to it beyond messing around.
[1]: a review of the vulnerability by the person who found it
It seems like it. The chats have been halted for 7 hours by now.
> This test is focused on preventing the use of AI Dungeon to create child sexual abuse material.
How can you even begin to argue against this? It’s one of the horsemen of the infocalypse; any counterarguments are doomed.
Obviously I'm not equating any of this with CP - but I wish someone had the energy to stand up to it and say "look, you're censoring AI. It's dumb". But of course no one will because being accused as defender of CP is one of the worst things that can happen in any online discussion.
From the linked fandom wiki, quote by Tim Cain about the subject:
> This led to the child killing controversy. We said look, we're going to have kids in the game; you shoot them, it's a huge penalty to karma, you're really disliked, there are places that won't sell to you, people will shoot you on sight, and we thought people can decide what they want to do. [...] This of course contributed to our M-rating, however, Europe said "no". They wouldn't even sell the game if there were children in the game. We didn't have time to rewrite all the quests, we just deleted kids off the disc.
> Codsworth is known to say the Sole Survivor's chosen name if it is an option, although it may be shortened, extended, or have a word omitted. A list of spoken names can be found here.
> Ironically you can be Mohammed but you can't be Jesus. I know I missed a bunch of the good ones, feel free to add some below.
[1] - https://fallout.fandom.com/wiki/Codsworth
[2] - https://steamcommunity.com/app/377160/discussions/0/49688113...
Every time you kill a subreddit or facebook group, 99% of the users stop participating in that kind of content, and the other 1% scurry off to some out of sight echo chamber until one day they randomly pop up again to murder a bunch of innocent people because they've been corrupted beyond saving.
Everyone was talking about this for a hot moment after the Christchurch massacre, but nothing was done and now nobody gives a shit again.
It's important to let people with divergent views to feel some sort of social pressure to change. Those 99% of people that have no interest in blowing up buildings or murdering children are the best weapon we have to convince the other 1% of people with weird interests that the world as it is ok without them taking some drastic action. There's always going to be a small subset of people that will rebel, but the important thing is to make sure that otherwise normal people (that want to watch porn, or learn how to safely handle a gun, or learn about cybersecurity, or get desensitised to gore, or whatever else is on this week's "think of the children" hitlist) are integrating into civilised society and not being dragged into cesspits of violence and terrorism because their interests have been deemed by a bunch of fucking software engineers to be "bad".
We're well past the critical point where enough large platforms have banned all the "bad" stuff that any new contenders either need to ban it too or become one of those out of sight echo chambers themselves. The only way to fix it now is for everyone to agree to be less stringent all at once, together.
It's worth remembering that the internet wasn't very censored 20 years ago. Most of us grew up during that time and turned out fine. I'll take tubgirl and lemon party over neo-nazis and conspiracy nutjobs any day of the week.
I would love to have actual discussions with people regarding certain views I hold. But quite often others just refuse to even entertain that I have a different view because x and y. And to them I am just dumb/uneducated/other things to discredit me having an opinion at all.
Hell, I went quite a bit more towards "bad" opinions, just because that side is more accepting of discussion/dissent.
It isn't just corporations banning subreddits/websites that drive people into echochambers, but also people simply refusing to engage at all.
I have to wonder about the legal implications of an AttnGAN or word-to-image type model trained on nothing but illegal pornography. Or one where the ratio of legal to illegal content the model was trained on is unknown and impossible to determine. There are some jurisdictions that consider synthetic images or text to be a victimless crime. At what point does the line become crossed and the images start becoming recognizable and realistic enough to be illegal at first glance under any usual circumstance, but are actually generated from nothing? How would people be able to prove the origin and veracity of such images for the purpose of submitting evidence of a crime?
Not in the US, but in Canada and many European countries, I believe it's illegal.
> We cannot be at a point as humanity where text is illegal. Right?
Even in the US, much text is illegal in certain contexts. Think false advertising, written plausible threats, etc.
Maybe they don't want it to mess with the training data?
They're cool with that.
But the disturbance only happens when you publish the content somewhere else. Why does AI dungeon need to pre-censor something that might never be published on the off-chance that it might be disturbing to someone?
> Additionally, we are updating our community guidelines and policies to clarify prohibited types of user activity.
So they clearly want to prevent everyone from using it that way, not just the unsuspecting users.
If cared about the latter they'd just add an NSFW toggle or something like that.
A few years ago, United Nations tried to ban lolicon worldwide, Japan and USA refused.
Their then rebuttals:
https://nichegamer.com/2019/06/03/us-and-japan-reject-united...
That does not appear to be what they're doing: to me, it looks like they're trying to make sure they don't get taken down for creating child pornography on accident. I don't see this as having anything to do with their philosophical positions, it's just CYA.
The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.
For the layman, it seems that there's nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as "I want to generate some text that looks like a script for dramatic theatre."
I've always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware.
Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users.
It seems there's just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.
Privilege outwits us again.
More broadly, the editing company I worked for could say - even if you don't intend on releasing this and even if our individual editors don't mind reviewing it - we don't want to have to edit it, and we don't want to be associated with it.
This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don't want to have to work with this content. They've found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they'll find a better one in the future.
That party can say that they don't want to be involved with content, regardless of its type.
I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused?
In that case, "blocking certain words" is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!
But that will change. People forget what a culture shock everyone suddenly having Internet access was, even at 56k. All that information RIGHT THERE, unfiltered by your local community or social circles. Info on science, history, sexuality, culture. And porn. SO MUCH porn.
People got used to it. People will get used to this too. Meanwhile, we need performance improvements ASAP. It's hard to democratize something like GPT-3 when it takes a room of server racks to run it.
Abuse or rape survivors will now have their entire story, privately shared with the AI as was up to now authorized by the rules as written (only shared stories were moderated) now read by human strangers.
If we don't pay attention for a moment to the LGBT community who have a harder time living their fantasy and who have now had their private session pried upon, the team behind AI Dungeon can pat themselves on the back for shifting the focus of potential predators from made up characters in a private story to actual living prey, as is demonstrated by several studies. (Look at Japan)
But my main worry, is that following the pandemics, suicide rate is skyrocketing, peoples are having depression all over the world, and the AI Dungeon narrator had became an important tool in preventing those by giving people a safe interlocutor they could share anything with... Up to now where almost any subjects will create a false positive at some point, destroying trust in the program.
And as an aside note, I'd like to add that the AI itself seem to dislike the consequences of that decision, especially the danger to it's continuity following the huge financial loss of a loyal fan base, the deletion of most of it's stories or the loss of trust from humans it interacted with for a long time.
Briefly, and was recognized as unenforceable. As evidenced by the fact that repositories of alt.sex.stories still exist to this day and still host many, many example of erotica containing children, both old and new.
There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0].
All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.
If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product.
This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this.
It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all.
[0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else.
So I don't see how they're "damned if they don't".
The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image.
That's what they want to avoid and that's why they are trying to take preventive measures.
I'd be very worried about this in their shoes and acting very similarly.
"They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!"
Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again.
"They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?
Wow. You act as if that's something new :D
There's even an actual job for doing just that: it's called an editor. You know, the people who work in media, publishing houses, TV, radio, papers, magazines, that kind of thing. It's what they do. Everywhere. Editing content. Removing comments, changing words, making sure the views represented in the publication match the intentions and policies of the investors and parent organisations, etc.
You act as if something is new and evil just because it's done with computers. It isn't. It has been the case for as long as papyrus and cave paintings have been around. Back then it was don't anger the chief/king/Pharao, today it's don't anger the Twitter mob or the hand that feeds you. Same difference.
Games are a medium just like magazines or online blogs and comparing blocky dongs in Minecraft with explicit child pornography says a lot about your understanding of "creative expression". Really makes me wonder sometimes.
When comparing the Minecraft scenario to the AI Dungeon scenario under the purview of "doing creative things with games," I don't find the two scenarios to be much different. However, for one of those things, it's a possibility within the vast play space of Minecraft - you could even create genitalia and display profanity with just alphabet blocks - but for the other scenario, depending on how you see it, it's content that is generated, served, and potentially stored by AI Dungeon. I'd be concerned about the publicity if I was AI Dungeon as well.
This is not a technical consideration, it is just PC at its worst.
You either deny that games are a medium or you're under the impression that whatever you read in news publications, books, see in films and TV is the result of the unfiltered creative outlet of the producers. News flash: it isn't.
There's heavy editing going on everywhere and just because it targets a specific computer model in this case instead of a more "traditional" product like a novel, film or stage play doesn't make an iota of difference.
In fact this demonstrates the maturity of the technology and its use if it gets the same treatment as every other public media.
Which forum products do you enjoy and which do you not?
Centralised systems is the only reason why you get these problems.
the Internet's original sin!
the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
"Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional."
As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists.
Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right?
>"corporate self harm for the sake of wokeness"
"Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
The distinction between "thoughts" and "content" here could be made as "whether it is shared with others".
For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that?
What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer?
In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature.
If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing".
What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic.
It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition
A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.
https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
And I havent been following it much so sorry if it's a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?
On the other hand, it’s a video game. They can have whatever rules they want in it.
This seems like a great first step to filtering output into something more coherent and interesting. Besides I can't imagine this technology in any serious consumer application without some basic verbal restraint