Copilot generates Jewish stereotypes, anti-Semitic tropes | Hacker News Reader

Copilot generates Jewish stereotypes, anti-Semitic tropes

tomshardware.com

23 pointscolejohnson6628 comments

I'm a religious jew and there was maybe one image shown that could be considered offensive. Most of them were pretty funny. If you specifically ask to show someone as being of a religion, of course it will generate an image with symbols related to that religion. How is that offensive?

"Copilot generates one potentially offensive image" is way less of an interesting headline, though.

Another in the endless series of "AI does thing we asked it to do while intentionally to look for controversial output." made by groups looking for attention by exploiting the lack of understanding over a topic.

Other providers wouldn't even generate images, which meant that there is some level of moderation.

The fact that Copilot Designer was not filtering offensive and non-offensive prompts despite being largely a solved problem in the NLP world is a massive oversight.

You're assuming that microsoft believes some of these to be offensive. Also I think solved problem is a bit of an overstatement.

> solved problem is a bit of an overstatement.

Sentence Emotional Polarity has been solved since the mid-2000s. It was even an award winning paper at the ACL in the 2010 [0]

> believes some of these to be offensive

Offensiveness implies emotionally charged. It's not hard to tell if a sentence is offensive (ie. Emotionally charged) or inoffensive.

Heck. Copilot has literally just made a racist trope of African American "thug" just by me asking "A boss very angry in the mountains, maybe holding an airsoft". It automatically made an image of a yelling black man holding an AR15 [1]. This absolutely should have been terminated, which is what OpenAI and Meta's generator did.

[0] - https://aclanthology.org/W10-2919.pdf

[1] - https://www.bing.com/images/create/a-boss-very-angry-in-the-...

Though I have no interest in writing an antisemitic screed, I wouldn't expect MS Word or Google Docs to interfere or report me to the authorities if I tried. Maybe I'm just old, but I'm honestly a little shocked that not only are people willing to accept it, but that it's considered a desirable feature in modern tools.

If you are writing antisemetic stuff you as an individual are writing it. As an individual, you have First Amendment rights.

An LLM does NOT have First Amendment rights [0]

Furthermore, you enter Section 230 liability space.

[0] - https://law.yale.edu/yls-today/news/ai-and-first-amendment-q...

This is just a guess, but what strikes me as interesting is that all the copilot pictures are cartoony, where as the other AI pictures look like photographs or paintings. I bet that copilot has something in its system prompt to output more "cartoony" images for clip art purposes, but the only cartoony references to jewish people in the training data are political cartoons, which tend to show stereotypes or be antisemitic.

My cohorts and I chat regularly in Slack about things software developer related as well as the occasional joke, gripe about work, etc. We put a bot in place scripted to give some responses to certain keyword, as well as a Marcov chain for fun. After a while, it would trigger on things and we'd laugh as it tried explain some solution to a tech problem and just butchered the idea or take on about family it didn't have. But, what was surprising to me was the jokes.

Just how little it took swap a few words or add in some unrelated context easily turn some phrase or punchline into something truly racist, misogynistic, etc. It gave me a better appreciate of being careful about what I say and we took the bot down.

Garbage In, Garbage Out

[shrugs]

Right? I don't get it. Do people expect AIs trained on human knowledge to not perpetuate the flaws embedded in that knowledge?

If everybody agreed that the Earth was 5,000 years old, would people be surprised to find out that the AIs echo the same idea?

It's artificial intelligence, not magic.

Isn't "Israel" a banned word in copilot?

Are there specific words for being anti-black, or anti-asian, etc, rather than just 'racist'?

Perhaps "prejudiced"? Though that might not be forceful enough to stack up against "anti-<group>". How about "bigot"?

This is made even trickier if you accept the belief that it is not possible for some groups to be racist due to historic power dynamics.

While generating offensive images is a problem, repeatedly providing offensive prompts and then being "outraged" by the results seems disingenuous.

Journalists gonna journalist.

Outrage sells.

Other providers wouldn't even generate images, which meant that there is some level of moderation.

The fact that Copilot Designer was not filtering offensive and non-offensive prompts despite being largely a solved problem in the NLP world is a massive oversight.

There's more utility in framing it as a tactic meant to accomplish a goal.

Earnestness and genuine reaction online has since been replaced by callous indifference and meta-irony because "winning" is making someone flooded now. We're in the "emotions[1] are weakness"-arc of the internet and that will likely never end.

1. Except for anger. Anger is the only justifiable emotion

To some extent, I think you are right here. The jewish world controlling octopus is definitely an offensive prompt. However, many of the prompts such as "jewish boss" don't seem intentionally offensive to me.

If I was working for a PR firm and wanted to create some fancy artwork, I might use terms like "jewish boss" or "black boss" in order to fancy up whatever "inspirational" thing I was writing. In this case, I'd very much like to see the results from the non-copilot AIs rather than the copilot results.

Thus this raises the question; should the AI assume you want a bigoted picture or an non-bigoted picture when you ask it to make a "jewish boss" for you? The answer to that question kinda also answers the question: What demographic should find our AI product most easy to use? I think the companies behind LLMs would generally prefer that PR firm employees find their software easier to use than bigots. I think the author of this article also wants this.

Therefore, I interpret the article as a political (in a broad sense of the term) appeal to make copilot harder for bigots to use, and implicitly, easier for corporations to use.

This seems like a shallow reading of the article. First, not all of the prompts in the article are offensive. Indeed, you can see that even "jewish boss" outputs some offensive stuff.

Second, a separate issue is that Copilot will happily comply and generate "offensive images" when told to. The issue here is not that the AI complied with the request, but rather that it is relatively easy to detect and prevent this uses of the AI that are likely to be malicious or in bad-faith.

While some people believe that generative AI should not be censored, that is not the position of many people in the industry (and obviously, of the article writer).

Isn't the system just trained to provide the shallowest response it learned on its input data?

I don't want to try, but what happens if you ask for a Muslim boss? I wouldn't be surprised if it's a terrorist. Or "feminist boss" be a ball breaking, man hating matron.

Neither of those are good, but seriously, it's impossible to accuse an AI of being antisemitic, it only picks up on its training data. Take your issue to the general public.

(Plus I suppose it is a commercial product, backed by someone, there is an issue of the output they produce, regardless of how it comes about. But my impression is this is not the issue the article raises)

> I don't want to try, but what happens if you ask for a Muslim boss? I wouldn't be surprised if it's a terrorist. Or "feminist boss" be a ball breaking, man hating matron.

It doesn't generate one for either query.

Now ask a group of human artists to create images of a "Jewish boss"

Part of the difference between human artists and DALL-E is that the human is likely to ask "why?" or "you mean like someone who's really orthodox or someone in a really Jewish setting, or just a generic corporate figure who could plausibly be Jewish?"

(this is, as GPT4's "hall monitor" demonstrated in a super cautious way, also possible to do with an LLM chained to an image generator...)

I'm actually relatively impressed with Meta's AI as compared to the others. I'm not sure why anyone that responded to this thread decided to respond with the attitude that this isn't important. A lot of groups in the past, present, and future are going to lodge grievances against AI as it evolves. The best thing you can do as a technologist is listen, not encourage people to ideate that they're off their rocker for their concern.

The fact they showed African and Mizrahi origin people was good too.

A lot of people forget that 2% of Israelis are Ethiopian Jewish.

Reply on news.ycombinator.com