Hacker News Reader

Comment by isoprophlex | Hacker News Reader

Wait until someone trains a model like this, for porn.

There seems to be a post-DALLE obscenity detector on openAI's tool, as so far I've found it to be entirely robust against deliberate typos designed to avoid simple 'bad word lists'. Ask it for a "pruple violon" and you get purple violins... you get the deal.

"Metastable" prompts that may or may not generate obscene (content with nudity, guns, violence as I've found) results sometimes shown non-obscene generations, and sometimes trigger a warning.

View on HN

I’ve thought about this and in fact porn generation sounds like a good thing?? It ensures that it’s victimless. Of course, there is a problem with generation of illegal (underage) porn but other than this, I think it could be helpful for this world.

i wonder if you could say “person who looks 15 but is 18”

If all of the child porn industry switched to generated images they'd still be horrible people but many kids would be saved from having these pictures taken. So a commercial model should certainly ban it, but I don't think it's the biggest thing we have to worry about.

If I had to guess, I'd bet they have a supervised classifier trained to recognize bad content (violence, porn, etc) that they use to filter the generated images before passing them to the user, on top of the bad word lists.

Exactly!

Most likely they just take the one from bing. Or, if they trained a better one, it goes vice versa sooner or later

This is mentioned, "content filters" are "blocking images that violate our content policy — which does not allow users to generate violent, adult, or political content, among other categories" and they "limited DALL·E’s exposure to these concepts by removing the most explicit content from its training data."

Honestly that part pisses me off. Who cares if their AI "makes porn" or something "offensive".

I suspect it's more a business restriction than a moral one. If OpenAI allows people to make porn with these tools, people will make a ton of it. OpenAI will become known as "the company that makes the porn-generating AIs," not "the company that keeps pushing the boundaries of AI." Being known as the porn-ai company is bad for business, so they restrict it.

The part I really don't understand the purpose of the threat "Further policy violations may lead to an automatic suspension of your account".

Why do that? Just refusing to run my query is sufficient. Who is harmed if I continue to bang my head against that wall?

Because it is a solid line (the policy rule) drawn across a fractal boundary (actual porn), and given lots of attempts you can find somewhere inside the line but across the boundary.

Stopping more than so many attempts makes this much harder / much less likely.

I tried the term "cockeyed" and got a TOS violation notice

Porn is cheaper to make, and probably pleasant. But, fantasy porn is not. I can see it sparkling a revival of fantastic erotica

Reply on news.ycombinator.com