Hacker News Reader

Comment by minimaxir | Hacker News Reader

tl;dr AI Dungeon was required to add additional content filters after it went too off the rails, which caused community backlash.

https://www.wired.com/story/ai-fueled-dungeon-game-got-much-...

View on HN

It was more than that. They also significantly downgraded the model. I didn't follow the details, but IIUC Dragon used the 175B directly initially, then I think they went down a model size at Open AI's behest. Finally, when Open AI announced pricing, AI Dungeon had to downgrade the model further.

But yes, the content filtering got out of hand too. I was initially fine with it, as its proposed intention was to filter out really illegal stuff, like underage content. I rarely hit the filter. But then they tweaked it at some point and I was triggering it constantly on otherwise benign stuff.

And they broke features constantly.

When I unsubbed the state of AID was broken features, micro-transactions, terrible AI model, and a glitchy, puritanical content filter.

The plus side is that it made the puny GPT-Neo model look like a godsend.

I believe they're currently using AI21's 178B Jumbo model for Dragon. Since they're completely off of OpenAI now, the content filter is much more lax.

really illegal stuff, like underage content

Wait, isn't this output just text? How is a text AI generating illegal content?

The content may not be illegal to possess, but if it's obscene, then it can be illegal to sell it, produce it with the intention of selling it, transport it, distribute it, and so on.

Could it really? I was under the impression that unless you incite someone to commit crimes (or confess to crimes), the story would be covered under "art" and therefore protected. It's just text after all. Where does the limit for "obscene" go?

It's very funny to imagine picking up a romance novel and making it illegal by scrawling "by the way the girl was actually 16 the whole time" on the inside of the back cover.

In the U.S., it's called the Miller test: https://en.wikipedia.org/wiki/Miller_test

Wow, I had no idea, that sounds really bad. The whole book banning debacle now makes sense and seems legal. That test seems to me to give way for courts to basically judge however they want, as all those three criteria are very subjective.

Also first time I hear about "patently offensive" and now I'm laughing. Thanks!

It wouldn't be illegal in the US, but it would be in many other countries.

It’s called thoughtcrime

https://www.quora.com/Do-thought-crimes-exist-in-U-S-law

Miller v California has been undermined so severely by subsequent decisions that it basically doesn't apply anymore.

Quora is quite unhelpful in any context.

It should be noted that the content filters were implemented really badly. Like, saying “my ten year old truck” would suddenly stop your story in its tracks, lots of things would trigger it. Plus, people became uncomfortable with the idea that the AI Dungeon employees were potentially reading your stories at any time.

Reply on news.ycombinator.com