Hacker News Reader

Comment by throwaway-kmc | Hacker News Reader

All this says is, she sent a mail to Brain Women and Allies that resulted in her immediate termination.

Does anyone else remember when she picked a fight with Yann LeCunn by deliberately misinterpreting his remarks to cast him as a racist?

Seems reasonable to assume she picked another fight. Reading the Twitter threat, it reads like she made demands in a way visible to a large number of employees (something about authorship on a paper, to skim the Twitter thread) and perhaps attacked Google while being a manager there (the emphasis on talking to non-management employees sounds like this). I assume the email will leak, this being Google we are talking about.

View on HN

Says she deliberately attempts to cast LeCunn as a racist is a very uncharitable framing, but putting that aside, it doesn’t actually seem reasonable to assume she picked a fight. We have no idea what was in the email.

How would you describe what happened? LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the bias in its training data. He was then excoriated as a racist by this woman and the Twitter mobs she incited, to the point where this Turing Award winner is no longer willing to engage with the public via that medium.

LeCunn made repeated attempts to explain his position, only to be virtually shouted over about he wasn’t listening to people of color because he pointed out that a model trained on White people learned to produce images of White people, and that if it had been trained on Black people it would produce images of Black people. It was a research paper, not a production system, and Gebru and her acolytes were much more interested in scoring cheap points than in having a serious conversation about either ML fairness or the merits of this particular paper.

Seriously, the primary sources can be read by anyone.

I'd never heard of this fight before, so I just looked it up. My interpretation is that he made that claim, Gebru responded by claiming that that was an overly reductionist view of things (given, presumably, the fact that biased assumptions could affect the coded assumptions as well as the training data), he maintained his opinion, and she felt frustrated that she didn't feel she was being listened to. I did not see her call him a racist, and I did not think either person was acting in bad faith; both seemed frustrated and unhappy that there was a disagreement.

Am I missing anything?

Here’s a good summary: https://www.google.com/amp/s/syncedreview.com/2020/06/30/yan...

I think it’s clear Gebru was acting in bad faith and LeCunn was baffled and trying to deescalate. Gebru responds to each attempt to deescalate by a reply designed to further rile up Twitter, without actually talking to any of the points LeCunn makes. Virtually all of her replies are some variant of “you are wrong, but I won’t say why” and “listen to me because I am Black”.

I wouldn't be shocked if Google saw this drama, saw that she was offering to resign, and was like "that's a freebie."

It seems like most of the other AI researchers, at least those mentioned in that summary, agreed with Gebru? So while Gebru may have been “acting in bad faith,” it seems that the consensus, at least as the article presents it, was that Yann was wrong.

Maybe you’ve noticed that almost everyone who thinks this was probably justified is using a throwaway, and everyone who is claiming it was unjustified is not. Only one group stands to lose their careers here.

The same is true of AI research Twitter.

I have to disagree. I read the article posted above and this strikes me as an excessively critical view. I don't see Dr. Gebru acting in bad faith although they are clearly frustrated with the exchange.

> Virtually all of her replies are some variant of “you are wrong, but I won’t say why” and “listen to me because I am Black”.

I don't see that in the summary you posted.

The discussion shows that Gebru is a racist to the bone because she viewed everything from the racism angle. LeCunn could be wrong by attributing bias to statistical bias in data, but how is it bigotry without evidence? Yet, oh no, LeCunn is a racist in Gebru's eyes, and in this strange American culture, Gebru could just call anyone or any opinion she didn't like racist or racism.

I'd say she deserved to be fired for she is a racist who's damning to a normal society.

Racism is discrimination based on race (from a position of power)

Whom did Gebru discriminate against, based on their race? No one.

That episode is unfortunate imho, but she only suspected racist rationales... Calling out racism does not make you a racist, just like calling something "fishy" does not make you grow fins and gills

Calling out racism does not make one a racist, for sure. Labeling anyone who disagrees with her a racist, without even a slightly plausible evidence, makes her a racist because all she can see is race. You're right that that's not the standard definition of racism. It's a fair game, though, as people like Gebru pretty much uses racism as a weapon however they like.

> Labeling anyone who disagrees with her a racist

Which she doesn't.

Most notably, people here have been raising concern about the discussion with Yann Lecun, but that was also just a civil, if tense, discussion.

You're better off not twisting the meaning of words to favor your argument, though. It'll just create more problems down the line

Fair point. "Any" is too strong a word that invites just one counterexample to get debunked. I'd say "in many cases" then, as demonstrated in her Twitter stream and her email. The words like "white men"( such a vile and irrational perspective), "oppressors", "racists" show up often. Case in point, a professor who tells his/her student that the student's life was easy, even though the student had to sleep only four hours a day. Lack of empathy, sure. It's racism without other evidence? Are you kidding me?

And her paper was rejected and her immediate reaction is to demand the company to reveal the identity of every reviewer? Yeah, right.

In Gebru's final thread on the topic (https://twitter.com/timnitGebru/status/1276326634195259392), she lectured LeCun how to give a proper apology, stated that he was engaging in "pattern of marginalization" causing "incredible harm", and instructed him to educate himself and listen to Black and Brown people before speaking to her again.

> LeCunn calmly and rationally explained that a particular model produced biased results, and that that this was due to the bias in its training data.

But this claim is certainly not necessarily correct, and he shouldn't have been so confident about it. Any part of a system can contribute to bias and that includes the model design, not just the data. If they want the system to work then they need to actually demonstrate it works, not just say it could with X change without testing that.

Though I remember people arguing it didn't work properly at the same time they were saying it was evil (because it contributed to surveillance). Which is odd because if you don't want it to exist, you shouldn't want it to work either.

I think it is reasonable for one of the founders and most highly respected members of the field to make confident claims about something that is as open and shut as this. The paper in question was training a network to upres faces. The researchers trained it on a dataset containing primarily White faces and released it on GitHub for reproducibility. Subsequent people used it on Black faces and found it make them white.

This is not surprising. Black faces and White faces are not the same data manifold. This is like training a network to upsample oranges, then running it on an apple and being surprised when the result is an unusual color and texture for an apple.

I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist? Their choice to work on super resolution? The fact that they released their work for reproducibility?

There is a discussion to be had here but it's not possible in public because of the escalation of the topic. This is also not the best thread to be having it.

Since everyone else is just saying "you don't get it" without explaining what "it" is, I will provide some brief avenues of exploration. Because they are brief, they will be coarse and imperfect, and aiming to give you directional assistance on the topic.

So with that massive preamble because I'm not seeking to argue, here you go:

* Pre-trained models already encode much of this bias and if you don't use them you won't get very far very fast

* Large available datasets also reflect these biases

* AI model performance is generally measured against this dataset, further entrenching the bias. If you're better on Indian face generation it will give you barely any benefit on most scoring methods

* Saying "the outcome is only biased because the data is biased" misses the fact that the performance is on the biased data

There's a little more to it but I believe that's the meat of it. Anyway, not too keen on arguing this. Just sharing because it took me some work to figure out what they were talking about and I wish someone had explained it to me so doing so here.

Thanks for the additional detail, but those all still boil down to dataset eventually, right? Those sound more like "yes, and" additions than the "no, you're racist" shitfight we saw.

In the interest of explaining (not arguing), I believe the distinction is sort of that the community at large recognizes performance only on certain metrics. Let's set aside the notion of whether something or someone is "racist" because that's a bit charged. We'll instead just talk about whether we're creating generally capable systems.

For instance, if I were to champion metric A which purports to measure performance on human faces but it really only rewards performance on Senegalese then models that do better in general may not be recognized for being better.

In an isolated sense this is not a problem. However if the mainstream is that all metrics that are taken seriously are dataset-biased then we'll have an environment where the models will be trained on biased datasets in order to be successful.

For instance if everyone uses LFW to determine how good facial recognition is, then Senegalese fine-tuned facial recognition tech will not be recognized as being good at facial recognition.

So the argument is that dataset bias is built-in to our approach to the problem. I, personally, think that this isn't malice. We need benchmarks to judge approaches against each other. Benchmarks always have a first mover advantage and a massive path dependence issue. The first benchmarks aiming for things on humans do not reflect humanity accurately. These benchmarks became standard among the community. To be taken seriously you have to do well on benchmarks that are standard in the community. Dataset bias is then natural in newer approaches because the approaches are judged against how good they are against the inaccurate (if you will) benchmarks.

So no one need be racist or anything for the end result of the field to end up being discriminatory.

I don't think a successful approach is to call people racist over this. After all, it isn't malice that guides them. The discrimination comes from the sort of historical accident that has North facing up on a map. And no individual is really racist. It's sort of like the Bechdel test: no movie is crappy simply because it doesn't have two girls talking to each other, but if very few movies have two girls talking to each other about something other than boys, then it makes you think "hmmm, why's that the case".

Wouldn't it be a lot more productive to lobby for inclusion of more samples into the performance measurement datasets than to get into super-combative twitter pissfights with people who aren't even opposed to that?

That's, like, not even culture war, it's just basic correctness of the reference datasets. If a fruit classifier was missing oranges, we'd just fix it and move on.

Haha, well, you know how it is. Everyone has their own opinion as to how to solve problems like this. Some people see their role as sort of fire alarms to say "Hey! This thing is happening!". You know, like how every thing has people at all stages in the process. Others might believe that this is one instance (the benchmarks being 'wrong') and that a better outcome would be that the people putting out this stuff are aware of ways that things can go wrong so they think about it in future (maybe this time it's these benchmarks, next time it's the other ones, and then maybe next time it's something inherent to the structure, whatever, it's a bit whack-a-mole to solve at the instance level).

Like, for instance, HN has people who will bring up privacy violations of big tech constantly. They see their role as making sure the conversation is happening. Not justifying this. Just aiming to understand it.

For my part, I prefer to take the approach you're talking about because I, too, think that the fastest path to this is getting the photos, labeling the photos, and then lobbying for inclusion. Ultimately, I think it's okay if things optimize fast for growth and then we fix up issues afterwards. So the people building the benchmark sets weren't able to get a set that's representative of humanity. Should they have waited till they could have done that? IMHO, no. Rapid release moves the state of the art forward and then we can put in all of these corrections as we move.

Then there's the question of whether all-humans dataset is a good thing or if instead a thing that is white-humans and another that is black-humans is better. Anyway, all said, my personal approach to this problem (if I cared about it a lot, which I don't) would be to say "Current benchmarks and training data available bias towards certain races. I'd like to build X/Y to solve that. Here's what I have so far" etc. etc. I think positive engagement like that yields better results because the vast majority of scientists actually aren't weird race supremacists and the vast majority of AI researchers will gobble up any more data you give them which is segmented and labeled differently, the greedy bastards :D

Some challenges that I can still see:

* Getting the data. Might not actually exist.

* Labeling the data. Probably needs some work.

* Getting it into the benchmarks. It'll invalidate old scores, so there's just a product adoption problem here. I don't know how the community handles newer releases.

As a last aside, I suspect that this conversation ended up the way it did because:

a. It's charged. It's race-based differing outcomes. That's a sensitive subject.

b. People feel unheard. This is natural. Like, this is not an 'interesting' problem. It's literally just a data error so the luminaries in the techniques part of the field aren't really that interested in it. And the techniques part is where the sexy is.

c. This sort of thing has a tendency to escalate. One side says "You're not listening to what I say" and the other side says "I'm not racist. I don't get why you're calling me that" and before you know it it becomes "You have to be racist to be ignoring me" and whatnot and de-escalation becomes impossible. Especially because everyone rewards the loudest on each side.

Honestly, I think it's quite interesting to observe and to understand as just a view into the human condition but we use AI models professionally in the GIS space and professionally we just don't go near this at all. No part of me finds it interesting to solve or to interact with the discussion in any way. I only sort of participated in this here because I think I managed some insight into what it is and I wanted to write that down because I wish someone else could have accelerated me into it.

Anyway, I think that's all the insight I have on the subject, so I'm going to just leave it there. Any more and I'll be ass-pulling.

I'd add that these topics tend to attract dramatic personalities, and that's what we saw here.

Do something productive and uncontroversial, or something unproductive and very controversial?

> I’m not sure what else there is possibly wrong here. Is the width of their convolutions racist?

Not that parameter specifically, but if you say the data is wrong then that means any other parameter might be wrong too. More data might mean the model is too small to fit it, or the hyperparameters might be wrong to train it, or you now have the wrong ratio of other phenotypes (let's say that instead of races…) in the training set and their results regress.

Also, if you're adding people with darker skin, that of course means the pixel values are lower. That matters for image processing code, things like SAD thresholds or noise reduction will work differently.

> Their choice to work on super resolution?

Superresolution is only useful as a toy and they should have mentioned that when they put up a live colab, yes. They added a disclaimer later on and it was good - there was a big issue where people were convinced this was somehow a surveillance technology because they watched too many TV shows, even though it literally can't work that way!

Occam's razor makes us think in this case it's the data, the simplest explanation.

The data isn’t a separate thing from the rest of the model (unless you’ve validated it works across different datasets). It’s all just a bunch of numbers that get jointly optimized.

>This is not surprising. Black faces and White faces are not the same data manifold.

in particular because features contrast (dynamic range) is lower for darker faces.

>Is the width of their convolutions racist?

Not width. As a result of the above mentioned lower contrast, the racist here is the sensitivity of the resulting Gabor filters produced by the training in the first convolution layers and the Gauss filters in the next layer's. I suspect to deal with that problem one would have to normalize dynamic range of the faces, i.e. it would look something like kind of lightening of the dark faces and/or darkening of the white ones.

Wouldn't that go both ways though? Why was Gebru so confident that the bias is because of something else other than the training dataset? She immediately assumed that there are other items contributing to the bias and devolved the quality of the conversation as well by making those claims in a frustrated/exasperated tone.

The most obvious and easiest way to clear this up would be for the original researches to train the model on a "Senegal" dataset as mentioned by LeCunn and see what the results are.

The whole point is that it never is trained on a "Senegal" dataset, is it? Why do we always see these errors where white-centric datasets produce white-centric results?

While it would be a symmetric situation a vacuum, we do not live in a vacuum. And acknowledging dataset bias by itself doesn't address the bias meaningfully. In practice, we often treat the bias as an exogenous factor when it is not, moving it outside the scope of our responsibility. But it is very much the product of our work, a reflection of our choices, values, and beliefs about what to prioritize. We can't abdicate our responsibility for it; the stakes are too high.

Well that is the whole point. The entire controversy could have been cleared up if the original researchers or even Gebru would have trained the model on a "Senegal" dataset, or a dataset diverse enough to their own liking and then make the findings public. That, to me is research. Throwing a fit on twitter because people won't buy that the outcome of this model is racist is not research.

I am not sure why the wording around this has to be so abstract.

The whole point is that working on more accurate methods for doing X is one direction of research and working on methods to correct for dataset bias in X is another direction of research. It's completely reasonable to work on either one of those separately, and it's unreasonable to demand (you may suggest, but not demand) that people working on the former direction fix the latter. Better algorithms and better datasets are both necessary but quite orthogonal.

As you say, it's a reflection of our beliefs about what to prioritize. If you're interested in developing methods of image upscaling that generalize better and preserve facial properties that were underrepresented in the training data, that's an interesting area of research, and you're welcome to work on it, but you don't get to demand that people working on something else prioritize this instead, they get to choose their own priorities unless you're paying them for a particular direction of research (e.g. like Google should be able to direct Dr. Gebru while she was working for them).

There's a responsibility to correct for bias when implementing models in production (e.g. if you'd be actually deploying it in Senegal, then there would be a responsibility to use a Senegal-appropriate dataset), there's a responsibility to acknowledge the bias of a particular algorithm if one exists - for example if it highly relies on contrast values which would be different for different types of faces, then that's relevant, because it's a statement about the generalization of the algorithm; but there's no proactive duty to "address the bias meaningfully", that's nice thing to do, but it's like charity - a voluntary choice to factilitate a social goal, but not a requirement or responsibility to do that. There's a responsibility to correct harms you caused, there is no responsibility to correct harms caused by others, that's a good thing to do, but not a moral duty.

Someone who invests a lot into charity work or addressing bias is doing a good thing, but it doesn't mean that people who are doing other things are abdicating their responsibility - it's never was their responsibility in the first place to fix social issues in the wider society. You can't simply point at random people and declare that they're going to be responsible for something that they didn't personally cause and where they did nothing wrong, that accusatory behavior is unacceptable, so naturally there's a backlash to people who try to assert personal responsibility of others without a basis to do so.

This is a great goal but the time and place for it is not in a super-resolution paper. There are specific conferences, benchmarks and projects where it can be achieved with more efficiency.

Does exogenous vs endogenous work in a vacuum either? It’s exogenous with respect to some things and endogenous with respect to others.

No. The easiest way to clear this up would be for Gebru to make a specific claim that was actually related to the model and include a pull request of her own modifications that she had tested to deliver superior results.

This would be called “research”. That is what she was hired to do at Google. For everybody of every race ethnicity orientation able status and background that would be honored to give their best effort at the opportunity to do research at google, acceptance of resignation was the right thing.

Of course. Or at the very least, she herself could have trained the model on a "Senegal" dataset and make the findings public.

My only point around her feud with LeCunn is that she threw a fit because people including LeCunn pointed out that it is not necessarily a race thing, and she wanted to give it a race spin without confirming that it was actually the case.

And I am not at all commenting on all this drama around her resignation/being fired without getting a look at the contents of the two emails: the one she sent to Brain Women and the one with her demands. But I know this: if I were to issue an ultimatum to my employer, I am treading dangerous waters and should be able to digest the outcome of that, including being fired.

LeCunn may not be correct, but attacking his character for be incorrect is even worse. Moralizing even scientific debate is the road to destruction.

She has been advancing her agenda for a while by strategically picking fights, and winning them because she's been on the right side of a political wave and knew how to ride it. Then she got blinded by her own success and overplayed her hand, a classic cautionary tale. Fun fact: many people, not necessarily managers, won't be shedding tears seeing her go.

It's not "reasonable" to assume "she picked a fight". It's not "reasonable" to fire someone for "picking a fight", whatever that is. It's awful how people are willing to be so charitable to Google here after doing the exact opposite in Damore's case.

People get fired for picking fights all the time. You say that's not reasonable, but what's reasonable about subjecting all the other employees at a company to somebody who picks fights?

It's not really about charity to Google. I just don't consider her Twitter feed a trustworthy source of information, given her general frequency of toxic Twitter drama and her extended, bad faith attack on LeCun. I wouldn't be surprised to learn that Google was in the wrong here, and if they are I hope she gets some kind of justice, but I don't think we can just take her word for it.

Yeah, after that episode I blacklisted her on my Twitter.

At least Google was consistent: it fired both people who were causing too much trouble.

(BTW This is actually the fun part of the Damore story: the guy think he is smart by explaining how "evolution" makes gender different but misses the ABSOLUTE main point of evolution which is that "to survive to an environement you must fit in it". He didn't fit in the google environement, he got fired.)

He's just seeking a different niche :troll:

Sounds like a win to me then.

Reply on news.ycombinator.com