Do you think Google's diversity team expected it would generate black nazis?
Probably not, but that is precisely the point. They're stubbornly clinging to principles that are rooted in ideology and they're NOT really thinking about consequences to the marginalized and oppressed that their ideas will wreck, like insisting that if you're black your fate is X or if you're white your guilt is Y. To put it differently, they're perpetuating racism in the name of fighting it. And not just racism. They make assumptions of me as a gay man and of my woman colleage and tell everyone else at the company how to treat me.
This is Google on one hand and the Internet on the other.
So probably not?
This is a debate between people who want AI to be reflective of reality vs. people who want AI to be reflective of their fantasies of how they wish the world was.
"What should we do about black nazis?" is a pretty basic question.
If they'd thought about that at all, they wouldn't have walked this back so quickly, because they at least would have had a PR plan ready to go when this broke.
That they didn't indicates (a) their testing likely isn't adversarial enough & (b) they should likely fire their diversity team and hire one who does their job better.
Building it like this is one thing. If Google wants to, more power to them.
BUT... building it like this and having no plan of action for when people ask reasonable questions about why it was built this way? That's just not doing their job.
Do I think that the teams involved were institutionally incapable of considering that a plan to increase diversity in image outputs could have negative consequences? Yes, that seems pretty clear to me. The dangers of doing weird racecraft on the backend should have been obvious.
Won't happen.
They'll hide behind the corporate veil.