It's kind of a testament to our times that the person who chooses to look at synthetic porn instead of supporting a real-life human trafficking industry is the bad actor.
In my experience with 2D artists, studying porn is one of their favorite forms of naked model practice.
There is a lot of (mostly non realistic) porn that comes out of art school students via the skills they gain.
Will it be a nightmare? If it becomes so easy and common that anyone can do it, then surely trust in the veracity of damaging images will drop to about 0. That loss of trust presents problems, but not ones that "safe" AI can solve.
Maybe, eventually. But we don't know how long it will take (or if it will happen at all). And the time until then will be a nightmare for every single woman out there who has any sort of profile picture on any website. Just look at how celebrity deepfakes got reddit into trouble even though their generation was vastly more complex and you could still clearly tell that the videos were fake. Now imagine everyone can suddenly post undetectable nude selfies of your girlfriend on nsfw subreddits. Even if people eventually catch on, that first shock will be unavoidable.
I don't see any evidence of that. What I see is that people who want to embarass and bully others are already fully enabled to do so, and do so.
It seems more likely to me and many of us that the bottleneck that stops it from being worse is simply that only so many people think it's reasonable or satisfying to distribute embarassing fake nudes of someone. Society already shuns it and it's not that effective as a way of bullying and embarassing people, so only so many people are moved to bother.
Assuming that the hyped up new product is due to swoop in and disrupt the cyberbullying "industry" is just a classic technologist's fantasy.
It ignores all the boring realities of actual human behavior, social norms, and secure equilibriums, etc; skips any evidence building or research effort; and just presumes that some new technology is just sooooo powerful that none of that prior ground truth stuff matters.
I get why people who think that way might be on HN or in some Silicon Valley circles, but it can be one of the eyeroll-inducing vices of these communities as much as it can be one of its motivational virtues.
I've always been able to say all sorts of lies. People have known for millennia that lies exist. Yet lies still hurt people a ton. If I say something like, "idle_zealot embezzled from his last company," people know that could be a lie (and I'm not saying you did, I have no idea who you are). But that kind of stuff can certainly hurt people. We all know that text can be lies and therefore we should have zero trust in any text that we read - yet that isn't how things play out in the real world.
Images are compelling even if we don't trust that they're authentic. Hell, paintings were used for thousands of years to convey "truth", but a painting can be a lie just as much as text or speech.
We created tons of religious art in part because it makes the stories people want others to believe more concrete for them. Everyone knows that "Christ in the Storm on the Sea of Galilee" isn't an authentic representation of anything. It was painted in 1633, more than a century and a half after the event was purported to have happened. But it's still the kind of thing that's powerful.
An AI generated image of you writing racist graffiti is way more believable to be authentic. I have no reason to think you'd do such a thing, but it's within the realm of possibility. There's zero possibility (disregarding supernatural possibilities) that Rembrandt could accurately represent his scene in "Christ in the Storm on the Sea of Galilee". What happens when all the search engine results for your name start calling you a racist - even when you aren't?
The fact is that even when we know things can be faked, we still put a decent amount of trust in them. People spread rumors all the time. Did your high school not have a rumor mill that just kinda destroyed some kids?
Heck, we have right-wing talking heads making up outlandish nonsense that's easily verifiable as false that a third of the country believes without questioning. I'm not talking about stuff like taxes or gun control or whatever - they're claiming things like schools having to have litter boxes for students that identify as cats (https://en.wikipedia.org/wiki/Litter_boxes_in_schools_hoax). We know that people lie. There should be zero trust in a statement like "schools are installing litter boxes for students that identify as cats." Yet it spread like crazy, many people still believe it despite it being proven false, and it has been used to harm a lot of LGBT students. That's a way less believable story than an AI image of you with a racist tattoo.
Finally, no one likes their name and image appropriated for things that aren't them. We don't like lies being spread about us even if 99% of people won't believe the lies. Heck, we see Donald Trump go on rants about truthful images of him that portray his body in ways he doesn't like (and they're just things like him golfing, but an unflattering pose). I don't want fake naked images of me even if they're literally labeled as fake. It still feels like an invasion of privacy and in a lot of ways it would end up that way - people would debate things like "nah, her breasts probably aren't that big." Words can hurt. Images can hurt even more - even if it's all lies. There's a reason why we created paintings even when we knew that paintings weren't authentic: images have power and that power is going to hurt people even more than the words we've always been able to use for lies.
tl;dr: 1) It will take a long time before people's trust in images "drops to zero"; 2) Even when people know an image isn't real, it's still compelling - it's why paintings have existed and were important politically for millennia; 3) We've always known speech and text can be lies, but we regularly see lies believed and hugely damage people's lives - and images will always be more compelling than speech/text; 4) Even if no one believes something is true, there's something psychologically damaging about someone spreading lies about you - and it's a lot worse when they can do it with imagery.
People believe plenty of just written words - which are extremely easy to "fake", you just type them. Why has that trust not dropped to about 0?
This has so far not been true of videos (e.g. a video of a celebrity from a random source has typically been trusted by laypeople) and should change.
Misinformation only works if it confirms what people want to believe already. That there exists or not exists such material is secondary at best. But well, that is off topic I guess.
Spend more time on Facebook and you'll lose your faith in humanity.
I've seen obviously AI generated pictures of a 5 year old holding a chainsaw right next to a beautiful wooden sculpture, and the comments are filled with boomers amazed at that child's talent.
There are still people that think the IRS will call them and make them pay their taxes over the phone with Apple gift cards.
Otherwise, why is AI specifically being targeted, other than the fear of new things that looks similar to the moral panics of video games.
In reality this is a disaster. The elderly and homeless people are already being left behind massively by a society that believes internet access is something everybody everywhere has. This is somewhat fine when the thing they want to access is twitter (and even then, even with the current state of twitter, who are you to judge who should and should not be on it?), but it becomes a Major Problem™ when the thing they want to access is their bank. Any technological solutions you just thought about for this problem are not sufficient when we're talking about "Can everybody continue to live their lives considering we've kinda thrust the internet on them without them asking"
A better analogy would be Nigerian prince emails, but only a tiny minority of people believe those... or at least that's what I want to think!
If you can generate synthetic images and have a channel to broadcast them, then you could generate way bigger problems then fake celebrity porn.
Not saying that it is not a problem, but rather that it is a problem inherent to the whole tool, not to some specific subjects.
This is the problem with these kind of incremental mitigations philosophically -- as soon as the actual problem were to manifest it would instantly become a civilization-level threat that would only be resolved with drastic restructuring of society.
Same logic for an AI that replaces a programmer. As soon as AI is that advanced the problem requires vast changes.
Incremental mitigations don't do anything.
I’ve wondered for a while if we just adapt to the point that we’re unfazed by fake nude photos of people. The recent Bobbi Althoff “leaks” reminded me of this. That’s a little different since she’s a public figure, but I really wonder if we just go into the future assuming all photos like that have been faked, and if someone’s iCloud gets leaked now it’ll actually be less stressful because 1. They can claim it’s AI images, or 2. There’s already lewd AI images of them, so the real ones leaking don’t really make much of a difference.
(I'm not a fan of the direction, but then I'm a product of stage 2).
I believe this worked with nudity and model when asked generated "smooth" intimate regions (like some kind of doll)
so you could ask for eg. generic president but not any specific one, so it would be very hard to generate anyone specific
It's only bad because society still hasn't normalised sex, from a gay perspective y'all are prude af.
It's a shortcut, for us to just accept that these social ideals and expectations will have to change so we may as well do it now.
In 100 years, people will be able to make a personal AI that looks, sounds and behaves like any person they want and does anything they want. We'll have thinking dust, you can already buy cameras like a mm^2, in the future I imagine they'll be even smaller.
At some point it's going to get increasingly unproductive trying to safeguard technology without people's social expectations changing.
Same thing with Google Glass, shunned pretttty much exclusively bc it has a camera on it (even tho phones at the time did too), but now we got Ray Bans camera glasses and 50 years from now all glasses will have cameras, if we even still wear them.
When Tron came out in 1982, it was disliked because back then using CGI effects was considered "cheating". Then awhile later Pixar did movies entirely with CGI and they were hits. Now almost every big studio movie uses CGI. Shunned to embraced in like, 13 years.
I think over time the general consensus's views about AI models will soften. Although it might take longer in some communities. (Username checks out lol, furry here also. I think the furs may take longer to embrace it.)
(Also, people will still continue to use older tools like Photoshop to accomplish similar things.)
Ironic, since so many furs are in tech, but we all have artist friends I suppose. People just forget that portrait painters were put out of business by photographers, and traditional artists were put out of business by digital artists. And so the cycle and the luddism repeats.
Another use case would be that retired porn actors could license their porn persona (face/body) to some AI porn company to make new porn.
I see big business opportunity in the generative AI porn.
Until we can train incrementally and distribute the workload scalably, it doesn't matter how open the models / methods for training are if you still need a bajilllion A100 hours to train the damn things.
Both sides view censorship as a moral prerogative to enforce their world view.
Some conservatives want to ban depictions of sex.
Some conservatives want to ban LGBT depictions.
Some women's rights folks want to ban depictions sex. (Some view it as empowerment, some view it as exploitation.)
Some liberals want to ban non-diverse, dangerous representation.
Some liberals want to ban conservative views against their thoughts.
Some liberals want to ban religion.
...
It's team sports with different flavors on each side.
The best policy, IMO, is to avoid centralized censorship and allow for individuals to control their own algorithmic boosting / deboosting.
I mean, a lot of moderates would like to avoid seeing any extreme content, regardless of whether it is too much left, right, or just in a non-political uncanny valley.
While the Horseshoe Theory has some merits (e.g., both left and right extremes may favor justified coercion, have the we-vs-them mentality, etc), it is grossly oversimplified. Still, a very simple (yet two-dimensional) model of Political Compass is much better.
The fun quirk is that there are similarities, and this model draws comparison front and center.
There are multiple useful models for evaluating politics, though.
It says on the wikipedia article itself 'The horseshoe theory does not enjoy wide support within academic circles; peer-reviewed research by political scientists on the subject is scarce, and existing studies and comprehensive reviews have often contradicted its central premises, or found only limited support for the theory under certain conditions.'
Look at the rules to win an Oscar now.
To cite a direct and personal case, I was involved in writing code for one of the US government's COVID bailout programs, the Restaurant Revitalization Fund. Billions of dollars of relief, but targeted to non-white, non-male restaurant owners. There was a lawsuit after the fact to stop the unfair filtering, but it was too late and the funds were completely dispensed. That felt really gross (though many of my colleagues cheered and even jeered at the complainers).
> I think it's impossible to ban 'conservative thoughts' because that's such a poorly defined phrase.
I commented in /r/conservative (which I was banned from) a few times, and I was summarily banned from five or six other subreddits by some heinous automation. Guilt by association. Except it wasn't even -- I was adding commentary in /r/conservative to ask folks to sympathize more with trans folks. Both sides here ideologically ban with impunity and can be intolerant of ideas they don't like.
I got banned from my city's subreddit for posting a concern about crime. Or maybe they used these same automated, high-blast radius tools. I'm effectively cut out of communication with like-minded people in my city. I think that's pretty fucked.
Mastodon instances are set up to ban on ideology...
This is all wrong and a horrible direction to go in.
It doesn't matter what your views are, I think we all need to be more tolerant and empathetic of others. Even those we disagree with.
Both sides need to get a grip, start meeting in the middle, and generally let each be to their own.
Platforms weighing in on this makes it even worse and more polarizing.
We shouldn't be so different and disagreeable. We have more in common with one another than not.
The points of polarization on each end rhyme with one another.
Stability AI, very understandably, does not want to be associated with "the porn-generation tool". And if, even occasionally, it generates criminal content, the backslash would be enormous. Censoring the data requires effort but is (for companies) worth it.
"Bad actor" is a pretty vague term, I think they are using it as a catch all without diving into the specifics. we are all projecting what that may mean based on our own awareness of this topic as a result.
I totally agree with your assessment and honestly would love to see this tech create less of a demand for the product human-traffickers produce.
Celebrity deep fakes and racist images made by internet trolls are a few of the overt things they are willing to acknowledge is a problem, and they are fighting against (Google Gemini's over correction on this has been the talk this week). Does it put pressure on the companies to change for PR reasons, yes. It also gives a little bit of a Streisand effect, so it may be a zero sum game.
We aren't talking about the big issue surrounding this tech, the issue that would cause far more damage to their brand than celebrity deep fakes:
Pedophilic image generation.
Guard rails should be miles high for this one.
Ronald Reagan was a bad actor.
George Bush wore out "evildoers"?
Where next... fiends, miscreants, baddies, hooligans, deadbeats?
Dastardly digital deviants Batman!