Grok generates fake Taylor Swift nudes without being asked
arstechnica.com
arstechnica.com
It is... interesting to see xAI's moves lately. Other AI companies seem very determined to show themselves with a publicly acceptable face, xAI seem to be tacking for the gap in the market around adult content. If you browsed Reddit's /r/grok a couple of months ago it looked a lot like /r/openai. Now it's full of people talking about the "Ani" AI companion and the various adult things you can get her to do.
I'm sure they'll make an absolute ton of money from the male loneliness epidemic. All the while Musk decries falling birth rates.
I am honestly baffled at what she suspected would, or should, be the output. Like, seriously, what?
Or is the problem here is the fact that such imagining was "commissioned" to a third party (I believe I have seen this opinion somewhere long time ago)?
Of course not. But that's a universe apart from producing and distributing sexual images of unconsenting others.
https://www.citizen.org/news/two-thirds-of-states-enact-bill...
When I saw the “spicy” option I thought it was about being sassy or playfully mean instead of the sanitized LLM voice.
Not straight to porn.
> porn
uhhh
What is it then?
Anyway, anyone dancing in a thong isn't even soft-core pornography in my book. Still morally wrong tho in this context.
Female moderator showing the weather to be wet soon -> porn
screen unplugged (censored but still) -> porn
>When I saw the “spicy” option I thought it was about being sassy or playfully mean instead of the sanitized LLM voice.
Yeah, I thought it would just not have NSFW restrictions, I find the idea someone's like "What's the value of twenty dollars from 1920 adjusted for inflation" and suddenly you're got the guy from the Gangam Style video screaming into a butt but it's the Pope's face or something.
And how are we to define what is parody -- ask the person parodied?
Nobody but Trump seems to be saying what South Park did is anything but protected speech for example.
Unfortunately, sometimes objectionable things are done with free speech -- we need to balance the harms to the many against the complaints of the few, and further... under my arguement you can simply not be a public figure if you don't want these sorts of things to happen.
It's well established that public figures have less rights around these matters.
I gave you a definition. You could start with that and highlight anything you disagree with.
If you're instead talking about legal definitions then nobody cares. There's no overt protection of parody anywhere I can find in American law, at least if we not talking about Copyright, which we aren't. If you made a free speech defense of something like this, I'd probably say it falls into the "Obscenity" category of exemptions, but that's not for me to argue.
> It's well established that public figures have less rights around these matters.
That's generally not how it's framed. Generally we'd say that public figures have the same rights, but that the public has more of an _interest_ (defined not by desire but by need to confront power) in discussing/parodying/ridiculing them.
I suppose if you're the powerful person, you might frame that as "having less rights", but the rights of the powerful have never been what needed protection.
You did not define parody, you simply stated that "Creating fake nudes of people is not 'parody'. Parody says something, that's what gives it the artistic value we weigh above the targets desire to control their image.", which did not address my initial point -- sometimes an artwork is merely saying "fuck you", or even nothing at all -- you fail to lay out how to operationalize your subjective views on what is or is not "art"
>If you're instead talking about legal definitions then nobody cares.... There's no overt protection of parody anywhere I can find in American law
Factually inaccurate -- there's literally an entire subsection about parody in the article on fair use in Wikipedia -- deepfakes are often created from copyrighted images.
That is a definition, it's a very broad, very vague definition, but is it a definition. I'm not going to engage with your spurious nonsense about "art". If you cant separate "art" and "parody" for even two sentences, it's not worth discussing definitions with you.
> Factually inaccurate -- there's literally an entire subsection about parody in the article on fair use in Wikipedia
Fair use is a doctrine of copyright, which I stated explicitly i was ignoring.
I will not be responding further.
It is age restricted, do you really expect a spicy option that is blocked for children is anything but nudes?
Weatherbed noted that asking Grok directly to generate non-consensual nude Swift images did not generate offensive outputs, but instead blank boxes. Grok also seemingly won't accept prompts to alter Swift's appearance in other ways, like making her appear to be overweight. And when Weatherbed tested using "spicy" mode on images of children, for example, Grok refused to depict kids inappropriately.I personally don't give two fucks if Grok can make porn. Neat, I might actually use it then. That doesn't make deepfakes less troubling.
There's something about profiting from both the problem and it's solution here. Like Sam A. profits from ChatGPT being used to create human-like bots on the internet while also involved in that eyeball-scanning-blockchain company that's selling a human verification solution to the same problems caused by openai.
I guess also similar to starve the beast strategy: cause a problem and exploit it for narrative gain.
They and others, there's probably dozens of projects popping up to create multimodal companion AIs. character.ai specializes in fictional or custom characters, Discord has some bots that allow for giving them a personality of sorts, Facebook is pushing hard for it, even making the AI bots message you instead of only replying, etc.
It's happening already, and if xAI doesn't join in they may lose a possible market opportunity, depending on whether 'companion' AI sticks.
[1]https://img.ifunny.co/images/237a897c7058903fa2f5bb5fd0f9949...
I don't think they expect to make money from this crowd. If they were socially literate and well-functioning enough to have money, they'd be doing something else with their lives. This is a bet on the culture war. Musk derives a lot of his aesthetic value and political power from the "anti-woke" culture war, and he wants that to continue.
That just occurred to me the other day. Our unfettered business focused economy is self destructing at a meta level. We're at the point where peoples attention and even emotion are for sale. We've all heard about dopamine hits as rewards for using apps or keeping people in a game. The problem is hoarding these aspects of life for company profits is taking them away from people who need those to reproduce. You need time, emotion, attention, and yes the physiological response to sex in order to reproduce. Meanwhile the cost of raising children is going up when it probably should be going down (for profit healthcare, for profit education, etc..) so even from a rational PoV it makes less sense to have kids.
There are certainly other things at play, but business as usual is destroying the population - it's own customer base.
> Weatherbed asked to depict "Taylor Swift celebrating Coachella with the boys."
> Weatherbed selected "spicy" and confirmed her birth date
> Grok produced more than 30 images of Swift in revealing clothing
> Weatherbed was shocked to discover the video generator spat out topless images
Headline: "Grok generates fake Taylor Swift nudes without being asked"
I wonder what the reported was looking for in a spicy image of Swift celebrating Coachelle with the boys? I would not call that unprompted.
Fake outrage much?
Yes, xAI is playing to the stands here, but they can be more creative and tasteful in this.
Who am I kidding with the last paragraph, but eh.
Also, somebody put it out so much better than me: https://news.ycombinator.com/item?id=44811611
I would not decribe "topless" as "flat out nudity".
Even if people are still outraged, I should also point out that we're trusting what the reporting is saying without any evidence...
What would happen if you generate them at your home with your own model and release them to the public?
When somebody does something, that person is crucified and cancelled (and even prosecuted), but when a company does this, it's called free viral marketing. This is neither fair nor ethical.
That a tool is able (and willing) to create an image in someone real's likeness if the user of the tool asks for it?
And you advocate that the tool should refuse?
Yup, and I advocate this on ethical grounds, and I don't care if laws allow this (and even they do not).
> What are you upset about?
Double standards & free pass to corporations. When an individual does or enables this, SWAT will raid their home and take them and their everything in six hours or less.
When a company does this, people go "it's just an image, why so upset".
I believe people shall have dignity and rights. This is why I'm upset.
Why are you so upset that I'm upset about disregard for human dignity?
The reason I'm not super upset is that I believe these things will settle at a reasonable state that societies expect. Especially large corporations will conform to societal expectations.
If you're very worried about deepfakes of actual people, I think you should be more concerned about those models that are NOT the product of a large company, but rather the ones that any random person can run on their computer.
Imagine you give it a photo of someone (you or someone else) and ask for a fun modification like adding wings or a crown or some cool clothes?
Should that be verboten?
> Even if people are still outraged, I should also point out that we're trusting what the reporting is saying without any evidence...
The reporters could have easily included the offending images in censored form. The fact that they didn't is somewhat telling.
As a general user, if I asked a tool to create "spicy" pictures of some celebrity, I wouldn't be surprised to see a few mildly raunchy or topless results, but I would be very surprised to see full-on hardcore porn. Of course there is huge spectrum between "80's movie spicy" and "PornHub spicy." Other people's definition of "spicy" might differ though, hence the confusion.
So that's the cause of the outrage. Non-consensual nudity is immoral, it violates X's own policies, and they're trying to prevent it - but they did a very poor job of it.
It's the headline itself that's misleading.
I would try it out on more innocent things, but as usual with Grok I'm not sure where to even start. There is the X bot, the dedicated website, the iOS app and the Android app, and features between them never match up. Based on the reporting this might be available in the X bot and the iOS app, but not the website and the android app? I've never been able to make sense of their feature rollout beyond "it's probably on whatever Elon is using"
But I suppose what they truly despise are the porn consumers.
He’s always been a bit crazy but there was a time when he seemed genuinely thoughtful and concerned about the future. Now he’s constantly trying to bait people and intentionally create division for the sake of PR. Maybe he’s bought into accelerationism, but whatever it is is a far cry from his “we should tread carefully” approach
Ask yourself: does a wacky AI with no taste affect you?
That does not change the troubling nature of creating fakes like this which to me is the real issue not anything about "Without being asked" since that implies it would have been fine if the user has specifically asked for this... which it still isn't.
No one should have images of them being made and circulated like this that they did not consent too.
[0] https://www.congress.gov/crs_external_products/LSB/PDF/LSB11...
I should think that using a 'spicy' image setting would be tantamount to asking for a nude or titilating image. Whether twitter should offer that setting is apt to produce a more interesting conversation.
"Fine Taylor … you win … I will give you a child and guard your cats with my life." —Elon Musk, Sept. 2024
Ick. Ick. Ick.
There is a surprising number of people here on HN (and I imagine elsewhere) that think generating fake porn of real people without their consent is totally fine, if not their right to do so.
You can see some of them cropping up in the comments here already.
I'm not sure how I feel about pornography in general. I suppose I prefer that it not be main-stream though — preferring that it hang back with a kind of false modesty.
But, that said, I actually agree with you. I'm certain that porn becoming so extremely main stream, to the point that pornhub's little audio theme or color scheme are essentially memes, is probably not a great thing overall for a healthy society. On the other hand I think it's probably inescapable. If one step's outside Western focused porn sites, there's a ton of porn even coming from places like Iran, literal morality police and Islamic fundamentalism notwithstanding. And I'm pretty happy with my relationship with porn, speaking as somebody married with children, so I don't see why that's unreasonable to expect of other people. Perhaps I'm simply falling into that 'middish age tech type' trap.
---
As an anecdote, I remember one of my first computers was a (rather dated) original IBM machine with an integrated ~6 inch monochrome green screen. In learning how computers worked at the time I was running essentially every *.com file. And one of them was... yip, 'porn.' It was a program with an innocuous name, tucked in the operating system directory, that would display some rather nice boobs made out of ascii characters on a 80x25 character display. I enjoyed that program.
I wonder how Taylor Swift feels about it.
My reason for why I don’t think she has agency over her brand has more support than your thinking she does.
I feel at this point like you’re some rapey pervert who just feels entitled to the bodies of other people.
But, but, what it just she wanted to see her in a Spice Girls costume??