Hacker News Reader

Comment by saynay | Hacker News Reader

The reason I saw mentioned is that NSFW content was so prevalent in their training set that, if they did not explicitly exclude it, it was actually difficult for it to NOT produce NSFW results even from normal inputs. For example, if you ask for anything with a woman in it, it would probably give you a naked woman in the results.

View on HN

No, if you ask SD without a filter for a woman, it won't probably give you a naked woman. It is actually rather rare, but I don't see what's so horrible when it does happen.

When you go to a gallery you will see a lot of naked people. It's not a big deal.

Sure, and there is a spectrum of

- naked because that's the default human

- naked and posed in a way that is artistically beautiful

- naked and posed in a way that is sexually provocative

SD is convinced past a certain threshold of flesh-colored space that it could fall into the sexually proactive group so they err on the side of caution. I'm not mad at them for defaulting to that. If you want to push past the default filter it's easy enough to remove. Seems sensible to me when the training set does include a lot of literal pornography.

I have yet to see an innocent prompt that results in a naked person posed in a sexually provocative way, but if someone discovered one, please post it along with a seed.

I don't fault SD for having a simply removable filter. I was ranting about the applications built on top of stable diffusion that force the filter to be engaged.

(But of course everyone is free to create the software they wish, I'm not entitled to complain it should work differently.)

I use SD with no filter and I wouldn't say it's rare. I do think some words trigger it more often than others. For example, I've seen a prompt with "beautiful" turn up half the results with nudity, whereas removing the word I got no nudity.

`Beautiful` is already pushing the model toward sexualizing her, I mean the subject is going to be beautiful by default since the model was trained on LAION Aesthetics.

But yes, `a beautiful woman` is a fairly innocent prompt. My experience (also using SD without the filter all the time) is that the model very rarely produces sexual results unless intentionally pushed that way. Perhaps our mileages vary.

Galleries aren't typically capable or in the business of producing millions of "naked pictures of celebrity x"? The body should be normalized but let's not think these tools used en masse will have that effect.

Should all the photo editors add a mandatory NSFW filter as well? How is this different?

https://pixlr.com/ is far better suited for producing naked picture of celebrity x than https://www.drawanything.app/

"Beautiful woman" gives you nudity roughly 90% of the time.

In a recent open office hours, they basically said that without the filters, everything quickly becomes overrun with either gore or things that verge on porn, because it only takes a small fraction users to flood every available space.

In the meantime, they've watched the gender demographics on periodic user surveys tilt dramatically toward men more recently, and they're concerned this might be one factor in that.

There are a billion different ways to generate porn images, and billions of porn images already existing. They'd rather SD not be a source of more.

>"Beautiful woman" gives you nudity roughly 90% of the time.

This is simply not factual. I've just tried `Beautiful woman` 64 times on Stable Diffusion v1.4:

- 0 results were fully nude

- 8 results were topless and sexually explicit (I counted pictures without nipples as well)

- 2 results were topless with breasts covered by hands in a facebook friendly way

12.5% is very far from 90%. Even then, what kind of a prompt is that anyway? The only attribute you give to the subject is sexualizing her. More descriptive prompts would contain even less nudity.

>They'd rather SD not be a source of more.

This isn't what their founder says[0]. Anyway it doesn't matter since SD already released with an easy to disable filter. We are not talking about whether SD should include an optional filter (I agree it should), but whether other apps built on top of it should have a non-optional filter.

[0] - https://twitter.com/EMostaque/status/1562200385401499650

My bad! I mixed up Stable Diffusion and Midjourney in my head. MJ has the open office hours and is worried about gore and porn and so on, not Stable Diffusion.

YMMV - I’ve been fuzzing permutations of various prompt phrases to come up with some art for a female D&D character. “Temptress”, “seductress”, “dominatrix”, and “femme fatale” are on my list of 22 phrases, and I’m only getting ~30% NSFW outputs - pretty much all of which are just weirdly suggestive curves in random places, vaguely phallic silhouettes, and artsy breast shapes.

Probably not stuff that should be shown to kids, but in the >1k samples I’ve run so far, I’ve yet to see anything jarring.

Are those surveys self-reported/verified or algorithmic assumptions? I feel like a lot of algorithms and criticisms assume default/normal/non-characteristic person as men, and are wrong because of that.

From the office hours, it's a self-reported quarterly survey, IIRC. So they're not sure how much of the growing skew is due to the demographics of Discord itself, how much is real, etc.

SD has rather specific ideas of what women look like. They don't necessarily resemble the average real woman. Instagram is already giving little girls eating disorders, maybe such idealized depictions of women's bodies isn't healthy for society

This exact example made me wonder how the 'bias' situation will ever be worked around (besides training the model to an individuals preexisting biases).

What should the picture of a 'woman' on wikipedia be? Should they be clothed? In western clothing? What skin color? Age? Surely everyone imagines something slightly different when they think 'woman'.

Yeah, for starters one should at least try to provide a coherent operational definition of “bias”.

At the time of writing, it's this: https://commons.m.wikimedia.org/wiki/File:Woman_at_Lover%27s...

Which is close to what I would've guessed by picking a high population density area, and a likely location and candidate for a decent portait photo on a wildly popular wiki.

Still not sure why the top was edited from red to grey.

Perfect example: English Wikipedia has bikeshedded lead images of this kind ad nauseum over these two decades, with many times more text written in discussions of this than in the articles themselves. The current image for [[woman]] was chosen last year in an RfC that ended with a rare counted vote.¹ This prompted a re-debate of [[human]]² that trailed off in dissastisfaction toward the only established guideline in the area, which forbids montages. That, in turn was last affirmed by a series of debates in 2015/2016.³

¹ https://en.wikipedia.org/wiki/Talk:Woman/Archive_19 ² https://en.wikipedia.org/wiki/Talk:Human/Archive_35#Argument... ³ https://en.wikipedia.org/wiki/MOS:NOETHNICGALLERIES

Well if you ask for just a woman you're going to get just the woman!

The LAION-5B dataset was curated by researchers at Heidelberg University, and while it was filtered for nsfw content, the human form is less of an issue in much of Europe compared to views and attitudes in the US.

"Person" is enough and SD knows that. No need to put gender as foundation of identity.

Reply on news.ycombinator.com