Hacker News Reader

Comment by qdot76367 | Hacker News Reader

Nah it’s just basic due diligence for releasing an open source app of this nature. Turning it off is a simple one line change, because everything is obviously named. This is not some high wall to scale.

As someone that manages nsfw open source projects, this move seems fine to me.

And actually kinda hilarious.

View on HN

If what StableDiffusion did was ask the user something like "The prompt you entered may result in the generation of content some people find objectionable. Are you sure you want to proceed?", then I would buy your argument.

As currently implemented (and the implementation took more work than a confirmation prompt would have!), it's an obvious attempt to control, rather than protect, users. They try their best to dress it up like a joke, but it clearly isn't.

It's open-source. Patch it out yourself. It's not hidden.

That's not the point at all.

It's the point of open source.

You have no point whatsoever. You're fighting windmills.

I disagree that the rick roll is inappropriate, but i also hate the argument that just because its open source, the programmers are above criticism. I think its reasonable people can dislike or even morally object to the decisions open source maintainers make (even if i dont particularly object to this specific instance)

They're not above criticism, but what they've done is a very small layer that's easily removed. You can do it, and release a version without it, in minutes. You can do it in less time than it takes to write a long comment decrying it.

It's not fundamental to SD in any way, and their suggestion wouldn't have worked.

I don’t read it that way. I think their intent is open to interpretation.

Fork it and fix it, see if your branch gains traction.

As currently implemented, it’s a hilarious Easter egg. I wouldn’t read too much into it.

They aren’t trying to “control” users, they’re trying to minimize the amount of bad press these AIs get. The approach they took was easier than your suggestion, because their system detects NSFW content on a per output image basis… so it’s easier to output some image (rickroll) rather than put up a prompt or even skip it completely (since they also generate a grid of the resulting images).

I should add that it’s also incredibly easy to comment it out.

Yeah, it's basic survival. Technology is inherently free and driven by porn, but sometimes that can be shocking to those uninitiated, so to "protect the children", restrictions are often placed to avoid alerting the horde and reverse progresses. I think those "enlightened" should construct and share such understanding that these are the actual reasons, so to avoid us all ending up in an actual dystopia where "protections" are enforced stricter than intended.

The content it can produce is illegal in many countries. It's not something they just decided to do one day on their own.

That would absolutely be more work than just subbing in an image and would neither have worked in the original implementation nor the various frontends it's been used in.

This is quite a classic HN kind of comment. Immediately assumes specific problematic intent and proposes a solution that doesn't fit the API.

Now instead users have to spend hours upon hours editing docker images to remove it. Much easier.

That's neither true not would it have been helped by making the original code require input during generation.

They put a simple check in, that tries to avoid returning nsfw images so that you don't get that back despite a more 'innocent' prompt. It's trivial to remove and is only part of the demonstration scripts, it is not part of the model or anything fundamental to it's workings.

From your previous comment:

> AI engineers starting to act like megalomaniac overseers who consider it part of their mission to steer humanity onto the "right" path.

You outdid your comment. You have an idea of how this should work. And you're trying to supposedly "steer" in a path of humanity. It's just projection.

This kind of preachery righteousness control tactics should sit elsewhere (fork it). The devs want to convey something is nsfw. They will do it however they want.

The implementation as it stands is actually easier the way they do it. They replace the image by scaling Rick to the image size. That means that they can still automatically put your output images in a grid without any extra code. They could have zeroed the pixels, but that could have led to confusion about if things were working or not.

It's silly and you're able to patch it out if you so desire. But most importantly, they released the model to everyone so it's way more open than OpenAI's DALL-E 2 (for better or worse). I don't think the argument that they're trying to control you really makes a lot of sense given how widely accessible this model is.

Absolutely. The limit isn't in what you ask for, it's in what's returned. The whole point is to try and stop the user being shown nsfw images despite sfw prompts, and since you can generate multiple results per prompt adding an interactive check on each one doesn't make sense.

What would the average 11 year old answer?

You know you can download the code yourself, right? You need not be reliant on their web interface if you're having these control issues.

It's also clearly in place because they don't have an age disclaimer, which would likely open them up to some sort of liability (or additional liability). Again, it's open source, and freely available, so like. . . this feels like a you problem, sorry.

Reply on news.ycombinator.com