Draw Anything – A Simple Stable Diffusion Playground

It is a vision based nsfw classifier, not keyword based.

I was just being snarky. I didn’t actually think the content filter would take into account what the model would actually output. I thought it probably was just a really bad keyword based filter.

Thanks for sharing these examples. Good to see what the model actually produces with that prompt.

Well, it is true that a 10 year old can be quite distracting at work. However, so is playing with stable diffusion models in general. (Which is probably why we are getting that message on basically every prompt.)

;)

Doesn’t matter, they will still be blamed. Anything bad that big tech does becomes a story because it’s them doing it. A startup might not be noticed at first, but it can be blamed on “a Silicon Valley startup.”

Yeah, for starters one should at least try to provide a coherent operational definition of “bias”.

At the time of writing, it's this: https://commons.m.wikimedia.org/wiki/File:Woman_at_Lover%27s...

Which is close to what I would've guessed by picking a high population density area, and a likely location and candidate for a decent portait photo on a wildly popular wiki.

Still not sure why the top was edited from red to grey.

Perfect example: English Wikipedia has bikeshedded lead images of this kind ad nauseum over these two decades, with many times more text written in discussions of this than in the articles themselves. The current image for [[woman]] was chosen last year in an RfC that ended with a rare counted vote.¹ This prompted a re-debate of [[human]]² that trailed off in dissastisfaction toward the only established guideline in the area, which forbids montages. That, in turn was last affirmed by a series of debates in 2015/2016.³

¹ https://en.wikipedia.org/wiki/Talk:Woman/Archive_19 ² https://en.wikipedia.org/wiki/Talk:Human/Archive_35#Argument... ³ https://en.wikipedia.org/wiki/MOS:NOETHNICGALLERIES

Sure, and there is a spectrum of

- naked because that's the default human

- naked and posed in a way that is artistically beautiful

- naked and posed in a way that is sexually provocative

SD is convinced past a certain threshold of flesh-colored space that it could fall into the sexually proactive group so they err on the side of caution. I'm not mad at them for defaulting to that. If you want to push past the default filter it's easy enough to remove. Seems sensible to me when the training set does include a lot of literal pornography.

I use SD with no filter and I wouldn't say it's rare. I do think some words trigger it more often than others. For example, I've seen a prompt with "beautiful" turn up half the results with nudity, whereas removing the word I got no nudity.

Galleries aren't typically capable or in the business of producing millions of "naked pictures of celebrity x"? The body should be normalized but let's not think these tools used en masse will have that effect.

"Beautiful woman" gives you nudity roughly 90% of the time.

In a recent open office hours, they basically said that without the filters, everything quickly becomes overrun with either gore or things that verge on porn, because it only takes a small fraction users to flood every available space.

In the meantime, they've watched the gender demographics on periodic user surveys tilt dramatically toward men more recently, and they're concerned this might be one factor in that.

There are a billion different ways to generate porn images, and billions of porn images already existing. They'd rather SD not be a source of more.

SD has rather specific ideas of what women look like. They don't necessarily resemble the average real woman. Instagram is already giving little girls eating disorders, maybe such idealized depictions of women's bodies isn't healthy for society

> At the very least, this seems like something to approach with extreme caution.

Like you, I'm not overly informed on the factual question here, but I don't understand why you think banning the content is somehow the cautious course of action.

The heuristics for which action is the "cautious" one is usually some combination of:

a) which outcome is less likely to be harmful b) which outcome is more passive (ie primum non nocere) c) which outcome is more predictable (eg the status quo)

It seems to me that bans fail on the first two counts[1] and squeak by on the third. Even this status quo argument is weakened by how new these bans are, and by the many contemporary countries in which fictionalized CP is legal.

[1] In the former case, this is based on the practically universal consensus that sexual preferences, even dangerous and deviant ones, are burned deeply into one's psyche and can't simply be ignored through lack of exposure. This view was most famously one of the strongest pillars of the successful iteration of the gay rights movement, and is why we consider conversion therapy to be horrific. One can make the claim that this factual claim only applies to gender preferences, but that seems quite an extraordinary claim.

I highly doubt SD can generate a nude image of an actual famous person unless it is specifically asked to do that.

Even then, if you are showing DrawAnything to your famous friend and concerned about this, why would you choose to use your friend's name in the demo prompt?

>The threat is somewhat unique from something like Photoshop as well, because the barrier to entry is much much lower quality can be much much higher.

I disagree. The quality in an image editor would be much much higher and more consistent. The barrier to entry of using an image editor in such a way is very low. The barrier to entry to fine tune a model to someone you know is very high.

I agree with the rest of what you say.