Comment by godelski | Hacker News Reader

Comment by godelski | Hacker News Reader

Parent Full thread

Regardless, I still saw it happen quite a lot. Maybe I was seeing the images under different LoRAs. But I'm extremely confident I saw the celebrities in bikinis or other sexy settings. So let's be serious, any sexy setting, regardless of nudity, is making a certain implication... And sorry, but your NSFW detector is not going to catch that stuff (and it is going to frequently fail on even nudes).

And did they apply the detector uniformly? Like you get the detector on Emma Watson, but does it apply to Hermine Granger? Because I definitely saw that situation reoccurring. Seemed to be extremely common with cartoon characters. Especially given the explicit nature of PonyDiffusion, mixing anime generation and realsm... Just turn of NSFW filters and go look at top models and top LoRAs. Nude content dominates the charts.

Let's also be clear, there is a gender bias. I'd wager quite a lot of money that there are far fewer JD Vance nude deep fakes than many female celebrities.

They had the NSFW detection dialed up to the point where it would flag a bit of cleavage as inappropriate for real person depictions. They were very on top of this issue and really wanted to avoid any bad press over "deepfakes".

Characters are a bit tricky. For example, Lara Croft could refer to the fictional video game character, or the character in the movie played by Angelina Jolie. They allowed the former, but not the latter. Or a model trained on the concept of "Hermione Granger" from fan art would be allowed, but not a LoRA trained on Emma Watson from the movies. That seemed like a pretty reasonable position to take.

I'm not sure what you're going on about. I logged in, and the 3rd model (1st non anime model) is CyberRealistic Pony. Without the filters practically every example is NSFW (>9/10 of the images). Are you trying to say that these images are not frequently based on real people?

I don't mean just trained on. I mean there's a ScarJo lookalike front and center.

If you're expecting the LoRA attached, I guess you can go with one of those "AI Characters" and that's at least feigning compliance. But come on, you can hide the LoRAs that were used too. But come on... the models themselves ingest a lot of porn photos and celebrity photos. You really think this is going to stop someone from just describing the celebrity and getting their nude? Don't be so dense. You aren't fooling anyone here

I can't find the ScarJo image you are referring to, so it either looks nothing like her, or it got reported and removed. Yes, people use NSFW models to generate NSFW content... What a shocking discovery.

What exactly are you proposing here that CivitAI didn't do already? Yes, people can scrub metadata from images. This is the exact same issue that image hosts have faced for decades - people can upload abusive or illegal images. The solution has been to do automated filtering, and have users report content that slips through. The solution is not to ban Photoshop because some people use it to make fake nudes.

Reply on news.ycombinator.com