Hacker News Reader

Comment by cypress66 | Hacker News Reader

> Reducing bias: We implemented a new technique so that DALL·E generates images of people that more accurately reflect the diversity of the world’s population. This technique is applied at the system level when DALL·E is given a prompt about an individual that does not specify race or gender, like “CEO.”

Will it do it "more accurately" as they claim? As in, if 90% of CEOs are male, then the odds of a CEO being male in a picture is 90%? Or less "accurately reflect the diversity of the world’s population" and show what they would like the real world to be like?

View on HN

If accurately reflects the world population then only one in six pictures will be a white person. Half the pictures will be Asian, another sixth will be Indian.

Slightly more than half of the pictures will be women.

That accurately represents the world's diversity. It won't accurately reflect the world's power balance but that doesn't seem to be their goal.

If you want to say "white male CEO" because you want results that support the existing paradigm it doesn't sound like they'll stop you. I can't imagine a more boring request.

Let's look at interesting questions:

If you ask for "victorian detective" are you going to get a bunch of Asians in deerstalker caps with pipes?

What about Jedi? A lot of the Jedi are blue and almost nobody on Earth is.

Are cartoon characters exempt from the racial algorithm? If I ask for a Smurf surfing on a pizza I don't think that making the Smurf Asian is going to be a comfortable image for any viewer.

What about ageism? 16% of the population is over sixty. Will a request for "superhero lifting a building" have an 16% chance of being old?

If I request a "bad driver peering over a steering wheel" am I still going to get an Asian 50% of the time? Are we ok with that?

I respect the team's effort to create an inclusive and inoffensive tool. I expect it's going to be hard going.

> inoffensive tool.

Wouldn't that result end up being like "inoffensive art" or "inoffensive comedy"?

Bland, boring and Corporate-PC.

To a certain degree, yes. They care more about the image of the project than art. Considering a large amount of art depicts non-sexual nudity yet they block all nudity, art is not their primary concern.

Some people claim to be emotionally "triggered" by images of police. Does that mean DALL-E should also start blocking images that contain police?

Being offensive is only one way to be interesting.

There are others, like being clever, or being absurd, or being goofy, or being poignant, or being refreshing.

Of the good stuff, offensive humor is only a tiny slice.

offensive to whom is the sticking point when it comes to comedy

it takes a special talent to please everybody

You know a surprising way to solve the issues you presented? You train another model to trick DALL-E to generate undesirable images. It will use all its generative skills to probe for prompts. Then you can use those prompts to fine-tune the original model. So you use generative models as a devil's advocate.

- Red Teaming Language Models with Language Models

https://arxiv.org/abs/2202.03286

The latter. Here's what we, a small number of people, think the world should look like according to our own biases and information bubble in the current moment. We will impose our biases upon you, the unenlightened masses who must be manipulated for your own good. And for god sakes, don't look for photos of the US Math team or NBA Basketball or compare soccer teams across different countries and cultures.

> Here's what we, a small number of people, think the world should look like according to our own biases and information bubble in the current moment.

You're being quite charitable. It is much more likely that optics and virtue signaling is behind this addition.

If I search for “food” I don’t want to see a slice of pizza every time, even if that’s the #1 food. I want to see some variety.

I think you’re jumping to quickly to bad intentions. Injecting diversity of results is a sane thing to do, totally irrespective of politics.

You are correct but that's not what anyone's discussing.

If I search for "food", the reasonable result would be to get images that represent food according to its actual proportions of real life. E.g. if Pizza is the most common food at 10% prevalence, 10% of the images should be pizza.

That's not what OpenAI are doing.

They are introducing crafted biases to create images that deliberately misrepresent what the world looks like, and instead represent what they believe the world ought to look like.

You also need some reason why diversity "of this" is important but not diversity "of that". Why is diversity of race and sex so critical, but not diversity of age, height, disability? Should a search for "basketball player" yield 1/2 able-bodied people and 1/2 wheelchair basketball players? Why?

Then try to answer where you came up with the categories you do want depicted. Why are the races what they are? Should "basketball player" include half whites and half black people? Or maybe split in 3, white/black/Asian? Why not Australian Aborigines, native Americans, or Persians - so we can divide into 6? If you don't add Indian people to your list then, is that racist against them? How did you decide what must be represented, in what proportions, and what's okay to leave out?

Yes the quality of surrealist generations went down with that change suddenly including gender and race into prompts that I really didn't want anything specific in. Like a snail radio DJ, and suddenly the microphone is a woman of colours head.. I understand the intention but I want this to be a default on but you can turn it off thing.

They literally just add "black" and "female" with some weight before any prompt containing person.

A comical work around to so called "bias" (isn't the whole point of these models to encode some bias?). Here's some experimentation showing this.

https://twitter.com/rzhang88/status/1549472829304741888

As competitors with lower price points prop up, you'll see everyone ditch models with "anti bias" measures and take their $ somewhere else. Or maybe we'll get some real solution, that adds noise to the embeddings, and not some half assed workaround to the arbitrary rules that your resident AI Ethicist comes up with.

Add after. So you can see the added words by making a prompt like "a person holding a sign saying ", and then the sign says the extra words if they are added.

Yeah actually, good call. The position of the token matters, since these things use transformers to encode the embeddings.

https://www.assemblyai.com/blog/how-imagen-actually-works/

How does it deal with bias that is negative?

Would only work for positive biases where if they actually want to equalize it then it needs to be adding the opposite to negative biases.

To counteract the bias of their dataset they need to have someone sitting there actively thinking in bias to counteract the bias with anti-bias seasoning for every bias causing term. Feel bad for whatever person is tasked with that job.

Could always just fix your dataset, but who's got time and money to do that /s

It's also funny that this likely won't 'unbias' any actual published images coming out of it. If 90% of the images in the world has a male CEO, then for whatever reason that's the image people will pick and choose from DALL-Es output. (Generalized to any unbiasing - i.e. they'll be debiased by humans.)

Imagine you're in South Korea (or any other ethnically homogenous country). Do you want "black" "female" randomly appended to your input?

If I was using this in South Korea, how is showing all white people any better than showing whites, blacks, latinos and asians?

You would presumably input “South Korean CEO”. DALL-E would then unhelpfully add “black” “female” without your knowledge.

I just tried it out and it looks like DALL-E isn't as inept as you imagined. Exact query used was 'A profile photo of a male south korean CEO', and it spat out 4 very believable korean business dudes.

Supplying the race and sex information seems to prevent new keywords from being injected. I see no problem with the system generating female CEOs when the gender information is omitted, unless you think there are?

Isn’t the diversity keyword injection random?

My point is that it is pointless. If you want an image of a <race> <gender> person included, you can just specify it yourself.

> If you want an image of a <race> <gender> person included, you can just specify it yourself.

I agree wholeheartedly. So what are we arguing about?

What we're seeing is that DALL-E has its own bias-balancing technique it uses to nullify the imbalances it knows exists in its training data. When you specify ambiguous queries it kicks into action, but if you wanted male white CEOs the system is happy to give it to you. I'm not sure where the problem is.

I don't think they "randomly insert keywords" like people are claiming, I think they probably run it through a GPT3 prompt and ask it to rewrite the prompt if it's too vague.

I set up a similar GPT prompt with a lot more power ("rewrite this vague input into a precise image description") and I find it much more creative and useful than DALLE2 is.

In their examples, the "After mitigation" photos seem more representative of the real world. Before you got nothing but white guys for firefighter or software engineer and nothing but white ladies for teacher. That's not how the real world actually is today.

I'm not sure how they would accomplish 100% accurate proportions anyway, or even why that would be desirable. If I don't specify any traits then I want to see a wide variety of people. That's a more useful product than one that just gives me one type of person over and over again because it thinks there are no female firefighters in the world.

Most likely this was something forced by their marketing team or their office of diversity. Given the explanation of the implementation (arbitrarily adding "black" and "female" qualifiers), it's clear it was just an afterthought.

It's also odd since you'd think that this would be an issue solved by training with representative images in the first place.

If you used good input you'd expect an appropriate output, I don't know why manual intervention would be necessary unless it's for other purposes than stated. I suspect this is another case where "diversity" simply means "less whites".

Will it reduce bias across all fields? Or only ones that are desirable? How about historical?

"A photo of a group of soldiers from WW2 celebrating victory over nazi CEOs and plumbers".

hardmaru on Twitter has examples. It’s the second, the one they would like it to be.

Reply on news.ycombinator.com