Hacker News Reader

Comment by miohtama | Hacker News Reader

It was driven by stupidy

It was driven by a need to remove biases (and likely real racism) from the models. They were trained on internet content

How is changing history and making a a group of objectively white people black "removing bias" ˋ? What it is is literally bias. Like thinking Pi could be 4. Removing bias ends with truth, not these crazy wonky results.

Do you think they deliberately trained the model to produce images of diverse Nazis?

It is clearly a byproduct of trying to correct an unaligned, bigoted model, and that is an example of overcorrection.

> Removing bias ends with truth, not these crazy wonky results.

Unfortunately there is an awful lot of untruth on the internet, if you hadn't noticed. This necessitates some correction through post-training.

> bigoted model

A what? What does this even mean?

It means a model trained on bigoted material.

A model where asking for a math professor results in an image of an asian man, asking for an engineer in an image of a white man, and asking for a criminal results in an image of a black man

If you try to remove that in the name of "diversity" or being "less bigoted" you quickly end up with racially diverse nazis

That's what they did.

> OpenAI invented a technique in July 2022 whereby its system would insert terms reflecting diversity (like “Black,” “female,” or “Asian”) into image-generation prompts in a way that was hidden from the user.

> Google’s Gemini system seems to do something similar, taking a user’s image-generation prompt (the instruction, such as “make a painting of the founding fathers”) and inserting terms for racial and gender diversity, such as “South Asian” or “non-binary” into the prompt

More links to primary sources, evidence, and official statements in the article at https://arstechnica.com/information-technology/2024/02/googl...

Lol fair enough, I stand corrected.

It seems then that their objective is to superficially increase the diversity of results, to avoid bad PR, rather than actually neutralising harmful and untruthful biases of the input data.

So you have a huge number of photos of white people in the training data set, but other ethnicities exist. So to make the otherwise white-biased dataset less biased, you try to e.g. add a hidden system prompt that whenever the user asks for a group of people (unspecified ethnicity), it may instead ask for "mixed ethnicities" or whatever.

Ask for a group of Nazis, and that's it - this is how models work. No "LGBTQ liberal" propaganda is needed to explain it. Unlike what Musk is doing.

The training data and the end result should reflect the real world. No you don't need to "add bias to fight bias", just keep it truthful. Don't force SF-brained forced diversity/inclusion/whatever, we know well some models do.

> How is changing history

If you're aruging about historical accuracy, but still want accurate looking generated images, I don't know what to say.

But to the technical point, A large part of the training corpus has biases that if left unchecked would cause PR based disasters for the company hosting it. ie the classic black teenager/white teenager.

Now as training of models is not an exact science, and neither is the fine tuning, its analogous to forcing a water balloon into a square box. Its possible but it has odd side effects when you get to the corners.

When making a _product_ you need to choose the least worse failure case. For grok it was for a long time, pandering to the ego of the owner. For Google, who is an advertising company, its about trying not to scare advertisers. This means everthing must be vanilla

If legit being truth-seeking even if it is a hard path means "pandering to someones ego" for you, whatever you say should not be taken too seriously.

You're confusing biases (stereotypes, bigotry, racism) with facts (2+2=4 isn't a bias)

No I'm not. Gemini spouting BS about the founding fathers being black is getting facts wrong because of its bias.

Political driven stupidity.

Reply on news.ycombinator.com