Google's Culture of Fear
piratewires.com
piratewires.com
This is both hilarious and sad
"Roughly, the “safety” architecture designed around image generation (slightly different than text) looks like this: a user makes a request for an image in the chat interface, which Gemini — once it realizes it’s being asked for a picture — sends on to a smaller LLM that exists specifically for rewriting prompts in keeping with the company’s thorough “diversity” mandates. This smaller LLM is trained with LoRA on synthetic data generated by another (third) LLM that uses Google’s full, pages-long diversity “preamble.” The second LLM then rephrases the question (say, “show me an auto mechanic” becomes “show me an Asian auto mechanic in overalls laughing, an African American female auto mechanic holding a wrench, a Native American auto mechanic with a hard hat” etc.), and sends it on to the diffusion model. The diffusion model checks to make sure the prompts don’t violate standard safety policy (things like self-harm, anything with children, images of real people), generates the images, checks the images again for violations of safety policy, and returns them to the user.
“Three entire models all kind of designed for adding diversity,” I asked one person close to the safety architecture. “It seems like that — diversity — is a huge, maybe even central part of the product. Like, in a way it is the product?”
“Yes,” he said, “we spend probably half of our engineering hours on this.”"
If true (admittedly a big if), it torpedoes the suggestion made by many that Gemini's misbehavior was simply accidental teething problems that were missed in testing.
In fact, Google does have a diversity problem, but it's the opposite of what the corporate apparatus thinks it is.
The hard left is a big fan of the belief that reality comes to reflect its symbolic representation, not that symbolic representations reflect reality. In such belief systems depicting something symbolically can make it come true. An example of this in earlier eras was the Sapir-Whorf hypothesis. This shows up as changing the race/gender/sexuality of people in art forms like job ads or on screen in an attempt to change job demographics in reality, but it can also show up in other ways. For example if you believe in symbolic determinism, then drawing Nazis is dangerous because doing that actually creates Nazis in reality (or in weaker forms of the belief, increase the probability of people's beliefs becoming more Nazi-like).
Remember that 50% of their effort went into stopping people drawing pictures of white men even when that would be highly realistic, because they believe that depicting us directly creates problems in the real world. For such people testing prompts expected to produce only white men would not only be risky in a careerist "what would HR say" sense, but it's perceived as risky in its own right! The very act of testing could make the world worse. That's the whole reason they're trying to stop users doing it in the first place! It's also why they constantly bring up physical safety even when talking about symbols.
So the Gemini team have ignored the actual feedback they got and decided that the error was making groups of bad people diverse. They're going to fix Nazis and Vikings to either refuse to depict them at all, or to show them as all white men (because those groups are bad), but ignore all the cases where groups of good people get incorrectly diversified. And of course they won't do anything about the text. This is all predictable given their beliefs about the nature of reality.
That leaves you with just override from above ie top management. And nobody sane is taking an even vaguely anti-DEI stance. So they seem a little stuck.
Won’t be the last major fk up we seen from them on this topic as a result.
> diversity architecture
Jikes
>“We definitely messed up on the image generation,” Brin said Saturday. “I think it was mostly due to just not thorough testing. It definitely, for good reasons, upset a lot of people.”
Mike Solana, the author suggests he may fire Sundar https://twitter.com/micsolana/status/1764723032042725531
>Megablock is a tool that allows you to mute a tweet, as well as block its author and everyone who liked it in one hit. Its creation was inspired by a tweet from a Twitter user, Mike Solana... https://www.makeuseof.com/how-to-block-everyone-who-liked-tw...
Google was clumsy here (in the same way that OpenAI was with DALL-E) but no amount of patches are going to fix the culture wars. Taking a text prompt and turning it into a picture is always going to be contentious. There will always be some amount of real bias and some amount of perceived bias on top of that.
And it’s incredibly cheap and easy for users to find real or perceived bias if they’re looking for it.
They could even put the forced-diversity-overdrive filter as the default, and for "real mode" display a disclaimer about it having biases that are reflections of all the garbage human biases that have been fed into these systems. Then when the political hacks of either team post pictures that are just obvious results of the filter they selected, they can be ignored.
I suspect the stopper is that these machine learning systems are being marketed as superintelligent "AI" which can solve all of the world's problems, and having to baby them with settings would be a recognition that they're actually just imperfect tools.
The possibilities are many.
New Musical Express (NME) is a British music, film, gaming, and culture website and brand. Founded as a newspaper in 1952, with the publication being referred to as a 'rock inkie'.
Right now, maybe, although they have staff photographers to shoot content by description - in the near future they and other magazines would want good AI image generation sites that can generate side content for pale white scottish bands and for Little Simz returning to roots in Nigeria: https://youtu.be/tvY31eN3gtE?t=61
If I'm writing a short story (let's say it's fan fiction for "Little Women") about a family in a New England village in the 1860s, images that include non-white people in them are going to look absurd and be ahistorical. Your assumption that the only reason to prompt for an image of an ethnically homogenous family is due to bigotry appears to reflect a negative and pessimistic view of humans. I would guess this view stems from spending too much time reading Twitter comments and forgetting that they are written by a tiny, non-represenative sample of the population that skews towards derangment.
I get that people will be really sad that they don't get to look at your AI generated 1860s white family because Google has made it illegal for all AIs to create images of white people. However I'll do you a solid: I happen to know a couple white families and I'll get them to pose for your short story. Just PM me the description and I'll hook you up.
Sometimes people want photos of families that look like their current reality.
I wouldn't want someone from a visibly different ethnicity when I'm looking for something like this.
Even black communities around me have visibly discernable differences. Where I live driving an hour might be enough. We aren't a monolith anywhere
When you take photos of your family do you rent others of a different ethnicity to avoid looking racist?
Which... Is the whole point of using AI. To be able to generate lots of different pictures based on your prompt.
That's perfectly reasonable.
Or maybe they just wanted a picture that looks like themself to post as a profile picture, for example.
Id hope that you don't support people posting profile pictures of themselves in blackface.
And in order to not do that, the AI needs to follow the prompt of the user.
Oh boy.
I doubt firing the CEO will do much at this point, the rot is institutional.
And will turn into investment fund once core function will be lost completely.
The question I am having trouble parsing is who makes the decision to become a pension fund at Google given their historical focus has been on technology. It's a significant pivot.
At a small closely held firm I can see the owners deciding to use the corporation as a vehicle for managing retirement funds (or inter-generational wealth transfer) but Google has a much more diverse base of owners with divergent interests.
I don't think anyone said Google has to become pension fund. They can become some generic investment fund: invest into stocks, other funds, bonds etc, or focus on tech investments.
The thesis I am fighting against is that all it takes to become a successful investor is a big pile of cash. There many many counterexamples.
Buffet and Munger at Berkshire Hathaway prove a different point: a sound investment thesis pursued diligently to enable compounding can yield very attractive returns.
(sorry, couldn’t resist…)
Yahoo is managing web properties and is essentially a media company (is paid by advertisers or other customers to deliver a well-characterized audience).
The people left are the hacks and the ones with mortgages.
Bad result:
Show me an Amish person eating chicken.
Better result:
Show me an awesome and amazing Amish person eating chicken.
My perception was that certain groups took the idea of diversity too far and made it into strict quotas. Certain speech is restricted. Promos went from being on merit to based on the skin color you were born into.
It's always astonishing to see ignorance and delusion so fully and grotesquely consummated in the mind of a person.
I, being a practicing Stalinist, believe that the only way to motivate the managerial class is with one way vacations to Alaska for under performance.
> Volunteer > The Seasteading Institute
Surely they could do the same for their image generation efforts?
Frankly are the "anti-woke" people really that fundamental to where the country is going? Do their complaints even matter all that much? On the US political side, they have been dead wrong in every political prediction since the 2016 election
1. 2018: record election year for the opposition
2. 2020: after a mess in handling covid the incumbent party is kicked out when most recent presdents get two terms
3. 2022: a supposed red wave never materializes.
4. 2024: ?
(This is not to mention all the underperformances for every "anti-woke/far right" candidate in special elections).
It sounds like they are just a loud minority.
Not for those that demand such politicized images, I suppose. But on all other metrics pretty much yes. At least their leadership said the responses were "completely unacceptable".
I don't get your reference to elections at all, it doesn't seem to make sense in context of your first question.
What other metrics are you justifying on? In previous comments I detailed my testing of Gemini 1.5 Pro and it shows significant improvement over Gemini 1.0 and Bard. My target is matching the ability of GPT-4 in my personal set of benchmark tests. It isn't there yet but going from 0 tests passed with Bard to 6 tests passed with Gemini 1.5 is a significant improvement in my mind.
[0]:https://news.ycombinator.com/item?id=39565128
>I don't get your reference to elections at all, it doesn't seem to make sense in context of your first question.
My claim is that the people complaining the most about what pictures Gemini are generating are part of one side of the US political aisle that is focusing on the content of these images and not any sort of technical improvements or failings.
I am led to believe they are a loud minority based on their election performance over the last seven years.
Calling gemini a "disaster" as the article does is too strong of a word given what I have seen with Gemini 1.5 Pro.
I believe this to be a vast simplification and a lacking perspective, but of course there are countless reasons why a certain political aisle would leverage such stories.
> Calling gemini a "disaster" as the article does is too strong of a word given what I have seen with Gemini 1.5 Pro.
"completely unacceptable" was a quote from the Google CEO on the topic of image generation. That the performance of Gemini is not the topic is part of that. The story isn't about any performance metrics of LLMs, it is about how it did generate some controversial images.
And that is indeed the larger question compared to how well it performs because there are more implications than a good test score.
And sure, the site and its cited twits could be cherry picking the worst examples, to make mountains out of molehills as political hacks tend to do. But it's talking about a notable product at a notable company, so unless those examples were outright faked there is still truth in its criticism.
Also I'm old enough to realize that 7 years isn't actually that long of a time, and one of the main reasons we get destructive spite votes like Trump is precisely people's resentment of this overbearing paternalism being pushed on them.
Yes you raise a good point, the tool as it stands is not useful but that was part of my point. Let me restate what I was trying to get at:
Less than year ago Bard was also totally useless. Sure as a text based tool, it wasn't as offensive to some people as this image generation tool is but the point still stands.
The question we should be asking: Is the tool completely useless to users and if so, how fast are they improving it so that it can be useful?
Useful in my mind is: Is the tool solving peoples needs? Clearly the image tool may not be doing so.
I personally feel that they have really done a good job improving their offering (Gemini 1.5) in such a short amount of time that I don't feel it is prudent to outright dismiss the possibility that if given some more time, we won't see improvement in their image tool.
>And sure, the site and its cited twits could be cherry picking the worst examples, to make mountains out of molehills as political hacks tend to do. But it's talking about a notable product at a notable company, so unless those examples were outright faked there is still truth in its criticism.
Yes criticism is absolutely fair...in my opinion it just looks too harsh to me given that it is such a new tool and we all know they are clearly behind. Our expectations are that it needs to instantly be at the level of their competitor when they have proven that they are behind.
>Also I'm old enough to realize that 7 years isn't actually that long of a time, and one of the main reasons we get destructive spite votes like Trump is precisely people's resentment of this overbearing paternalism being pushed on them.
Well I can't sit here and tell you how the future will play out. There are certainly many unknowns and yes a spite vote like Trump (and even Bernie) comes from a place of serious contention with the status quo. Reality is not playing out like that at this moment though. From a massive high of both right wing populism in 2016 to a massive rise in left wing populism ion 2018 we are also seeing a decline in the populous right/left wing. The online media landscape for both sides started shrinking after 2020's election, there is no real long term leader on either side on the horizon(Trump is on his way out and Bernie had two chances, now he too is at the end of his life), increasing amounts of infighting is eating both the left and right wing from the inside out and this year the left wing has been reduced to defending what little gains they did make in 2018/2020 while the far right has lost so much to the middle since 2016.