Hacker News Reader

Comment by bitshift010 | Hacker News Reader

Chief AI Scientist at Facebook, Yann Lecun, quit Twitter after a back and forth with this particular AI researcher - https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...

>I’m always amazed at how people can continue to do thing after thing like this and then turn around and ask me for some sort of extra DEI work or input. This happened to me last year. I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google (which is when they backed off--before that Google lawyers were prepared to throw us under the bus and our leaders were following as instructed) and the next day I get some random “impact award.” Pure gaslighting.

I think if you threaten to sue your employer, the employer isn't too much at fault and just covering their own neck when you later threaten to resign and they take you up on that offer.

View on HN

The tone of her replies is regrettable:

Yann: “ML systems are biased when data is biased"

Timnit: “I’m sick of this framing. Tired of it. Many people have tried to explain, many scholars. Listen to us. You can’t just reduce harms caused by ML to dataset bias. Even amidst of world wide protests people don’t hear our voices and try to learn from us, they assume they’re experts in everything. Let us lead her and you follow. Just listen. And learn from scholars like @ruha9 [Ruha Benjamin, Associate Professor of African American Studies at Princeton University]"

I don't like engaging this kind of discussion but have to call out the "I'm so tired" bit every time I see it.

But isn't it commonly accepted that ML systems reflect the bias in the data, and that it's hard to predict and control the bias in advance?

Her point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data.

This is a consequence of the fact that it's not possible to do generalization, and thus learning, without bias to begin with. What an AI researcher does when deciding which architecture is essentially tuning the bias in the generalization system in order to produce better results, which is not in and of itself bad, and is indeed unavoidable.

By bias, I mean really anything that means that the function g() is used over the function h(). In practice this can be anything from the seed for the random values, to the choice of activation function, to the architecture of the network, and so on.

If the architectures are selected for their performance on dataset y, then it is possible that even when trained on dataset x, they retain some of the bias of dataset y, because their generalization bias, which is architectural, was tuned for dataset y. This is, admittedly, a theoretical result, and it's not clear that it applies in every case, but it is a solid point.

For that reason the point that benchmarking against non-biased datasets is important was made, and thus that there is more to bias than the final dataset used for training during deployment. Therefore, it's not only a responsibility of the engineering community, but also of the scientific community.

I've posted it somewhere here already, but here's a nice paper that explains the point that there is more to bias than simply the dataset : http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf

Interestingly enough, this is true of all learning systems, including humans, and is a proof of why it's impossible to be completely unbiased in really anything.

> Her point is more than that. I think that she was trying to say that ML systems have an inherent bias that is independent of their data.

In general I take these types of tropes to be cloaked (politically correct) way of saying "fuck x group".

Eh. I think there's some legitimate concern that AI isn't fair to marginalized groups for a variety of reasons.

Dataset bias is only one way this occurs, though. Even if you include more colored faces, you still may be taking pictures with a camera that doesn't capture as much contrast. Or even if you retrain, you may have chosen model structure to optimize behavior in a biased dataset. Or... if you're considering the cost of misidentification by a facial recognition system used by, say, police, you need to be sure that you take a perspective that applies to society as a whole and not to your own interactions with police.

It's perfectly reasonable to say "AI is worsening the experience of minorities in various ways" and to differ with responses that assume with slight additional care in curating datasets that the problem will be completely solved.

One of the huge questions is : who takes responsibility for the issues with the black box learning systems ?

In France, since you have to be able to explain in plain words the algorithm that was used to help to make an administrative decision, it indirectly makes these black box learning systems illegal.

(And yet they probably already have been used by the police?)

Now, the naive response is that the engineer that implemented a system should take responsibility for any harm done by the system - and I guess that if these engineers could lose their license to use a computer and face jail time, they might also start to put pressure upstream on researchers to provide better systems ?

I'm not sure I understood your comment correctly, so I'll apologize in advance if I misunderstood.

That statement is contained in a paper I linked that dates from 1980. I'm pretty sure it's factually correct, because I'm trying to make nothing but a demonstration that an idée reçue, that ML systems are only based due to their data, is incorrect.

There's really no cloaked message behind it, and I'm not sure why one would think there is (hence the disclaimer). I think it's a pretty uncharitable interpretation.

> That statement is contained in a paper I linked that dates from 1980. I'm pretty sure it's factually correct, because I'm trying to make nothing but a demonstration that an idée reçue, that ML systems are only based due to their data, is incorrect.

That's reasonable. I may have taken the initial text in the wrong context. Thanks for clarifying.

To expand on my prior comment of "Fuck x group", perhaps the tone of prior tweets by Timnit Gebru short-cut me mentally, namely this one: https://twitter.com/timnitGebru/status/1274809418475950080

> Even amidst of world wide protests people don’t hear our voices and try to learn from us, they assume they’re experts in everything. Let us lead her and you follow. Just listen. And learn from scholars like @ruha9 We even brought her to your house, your conference.

It strikes me as very unprofessional, and frankly I believe her anti-white bias shone through in that instance.

Racism is racism, even for "good" purposes.

I think there is a bit of a double standard here. I try in general to be as charitable to both sides of an agreement possible. If your standard for uncharitability is such that the linked tweet is a proof of racism, I don't understand why one would take a tweet claiming against overwhelming evidence at face value.

Personally, I don't think either indicate racism. I see the link you tweeted as an excessively abrasive response at a dismissive and reductive understanding of an issue, bias in learning system, despite research old and new. At the same time, I see LeCun's dismissal as a genuinely held position in error.

To see the first as racist but the second as okay seems to be a case of being much more charitable to one side than the other, though I understand how it may look this way.

Whatever dude. You people are delusional. Stop bending over backwards to defend dumb ideas. To do so otherwise opens you up to mockery and ridicule of your otherwise substantive thoughts.

What "dumb idea" am I defending?

It's "commonly accepted" in that it is a well-known possibility and, therefore, something that practitioners should check for before publishing, using, or selling their models, yes.

It's not "commonly accepted" in the sense of throwing one's hands up, saying "there's just no way we could find enough pictures of black people for our AI to no longer confuse them with Apes" and move on.

Sorry from her rebuttal it seemed like there were other ways than just more and better labelled data. Is this correct ?

I don’t think that for a person who lives in a world where AI shops routinely sell software that discriminate against minorities, not just in ways that are explained by poorly thought out datasets, saying you’re “so tired” is at all wrong. This is the exact research that Timnit is an expert on, and these are systems that have real world consequences for minorities. It’s easy for people who are not minorities to say you should always a be polite but frankly I think she’s being extremely polite here given all of the circumstances.

I bet there is research showing how racism and discrimination literally tires out the people subjected to it. And now you have people saying they're not allowed to express that.

Indeed. reminds me of this speech by Jacob Blake’s sister: https://youtu.be/Ijp85on6UHY

“I’m not sad. I’m not sorry. I’m angry. And I’m tired”

Nobody said it was disallowed. The grandparent said he called it out as a lazy rhetorical move. It's certainly true that oppression can be tiring (does that point need research?) It may equally be true that some people overclaim on the extent of their oppression, and the mental health harms they are suffering from it, and even make a profession of doing so.

People are just tired of assholes who only blame the world and never take responsibility.

How are these AI systems affecting people? Is there an example application?

I find it strange to take issue with the "I'm so tired" line. It would seem to be entirely appropriate for circumstances where you see an argument that you have replied to many times but that completely ignores your reply. It also just doesn't strike me as particularly hostile or profane. Quitting twitter and making a show of it strikes me as a far more emotional reaction, but nobody here seems to take any issue with that?

In this case, the issue is that "it's the dataset" is attacking a straw man: nobody is accusing ML practitioners to be explicit racists actively conspiring for their models to be racially biased. Everyone understands that it is (often) a result of the underlying dataset.

But, the argument goes, understanding the genesis of such bias isn't enough to excuse it. At least not when those models are put to actual use in, say, law-enforcement contexts. Or credit agencies, or hiring decisions, or really anything.

If your dog happened to bite any red-haired kid it came across, it doesn't matter that you think red-haired kids are just as good as others, or that the dog was once mistreated by a red-haired kid. You're not going to allow the dog to get near any red-haired kids. And if you do, you'll face charges of at least negligence when some red-haired kid gets mauled.

To tie this back to AI, it would mean that as long as your models produce racially biased results, they simply aren't ready for deployment or publication. Go find better data until your work is no longer liable to inflict harm on anybody.

That's the standard for all other industries: if your car is shown to reliably kill pedestrians below a certain height in crash tests, you go back to the drawing board. Adding a sticker "don't operate car in the vicinity of children" isn't enough.

You can disagree with that reasoning (although you'd be wrong). What you can't do, as a leader in AI, is to pretend to have never heard of it, or that it is so obviously wrong that it warrants no reply whatsoever.

> have to call out the "I'm so tired" bit every time I see it.

so... I guess you're tired of it?

> so... I guess you're tired of it?

Yeah GP is modeling exactly the behavior they're railing against. Claiming that something is wrong without any practical insights for how that thing might be better.

It's not exactly the same behavior - it's along the lines of "can't tolerate intolerance" (meta).

> so... I guess you're tired of it?

I'm not.

Many other rhetorical moves have a lot of politics to muddle through / implicit assumptions that can be disavowed in bad faith, etc. I don't like to engage in that kind of discussion at all.

But the "I'm so tired" bit is both condescending (and it takes... something... to condescend to Yan LeCunn) and evasive. The end result is authoritarian -- look man, this is decided and settled, you haven't kept up.

Calling this out is interesting because it evidences on the contrary that the discussion isn't settled at all, and that she's unable or unwilling to frame it in a convivial manner that's conducive to progress.

I sit out of conversations I'm tired about.

> Many other rhetorical moves have a lot of politics to muddle through / implicit assumptions that can be disavowed in bad faith, etc. I don't like to engage in that kind of discussion at all.

But isn’t what you described so vague and subjective, that you can apply it to pretty much anything?

IMHO it’s more honest to say “I sit out of conversations I don’t like”, which is an entirely fair and understandable decision to make :).

It’s hard to like something that makes you tired when it’s not one of your kids.

Yann was (actively or unintentionally) refusing to participate in the same conversation. His response was the "facts don't care about feelings" one. It's not wrong but it is a thought terminating cliche.

It is. Yann also seems blissfully unaware of how sampling bias issues have always been dealt with (eg the Heckman correction model) in traditional statistics, where “getting more data” is not an option.

This post is a really good explanation, and I wish it had been in that Twitter thread.

The comment by the first John in the comment of the article here [https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...] adds a lot of insight too (starting with "As I understand, the social and structural problems are that AI learns the social and structural biases that are already present.").

> To tie this back to AI, it would mean that as long as your models produce racially biased results, they simply aren't ready for deployment or publication. Go find better data until your work is no longer liable to inflict harm on anybody.

Well, the disagreement is about publication. Yann LeCun himself said that "The consequences of bias are considerably more dire in a deployed product than in an academic paper."[1]

Why would it be wrong to publish research based on biased data?

[1] https://twitter.com/ylecun/status/1274790777516961792

What Yann said about data bias is a fact. He wasn’t dismissing your concerns nor distracting from it. A lot of people tried to extrapolate what Yann meant, but from the way I see it, if more people learn about the cause of this racial bias, the more people will be inclined to use fairer and diverse datasets in their future ML research.

The point is that she has an expectation of Yann to go further than just stating a fact.

I assume she thinks that Yann has enough intellectual capacity to improve situation of biased ML but doesnt execute. Further, my guess is that the reason why he doesnt execute is not transparent to her, hence frustrating.

Although that is a bit overreaching of an expectation in my opinion, I agree it would be good if Yann was a bit more explicit.

I don't think that's what she meant. You are filling in the blanks that she should have filled herself.

One of Jenny Holzer’s truisms is: you owe the world, not the other way around.

Isn’t Gebru an “AI researcher”, after all?

This is probably the core of the conflict there - there are researchers that aim to remove technical bias from technical systems and expect to stop there, not going into social issues of how the systems will be applied, as they consider that (e.g. solving the biases in the society) as a separate issue.

Dr. Gebru, among others, expects and insists that they do go beyond that and try to get involved in the social consequences of tech. They refuse, ergo conflict.

>I assume she thinks that Yann has enough intellectual capacity to improve situation of biased ML but doesnt execute.

Yann has no obligation to do so on Twitter, and shouldn't be called out by coworkers publicly regarding that decision, in my opinion.

> The point is that she has an expectation of Yann to go further than just stating a fact.

Yann was talking about a specific model, she was talking about ML & society on a grand scale. Can't we talk about specific things now? Why did he have to apologize for it?

My feeling is that Timnit is using people, she guilt trips, humiliates them online, makes a big fuss and it's all to whip up support in her crusades. She wants to win the public opinion by any means, so she needs famous opponents to pile on, such as Yann and Dean (who have universal admiration in the field). She's always playing the victim. I find this kind of behavior toxic, it degrades dialogue.

"It's December" is also a fact. He chose to mention a different one. Why, if not as an explanation / excuse for the model's behaviour?

Every practitioner already knows that these models behave according to the data they are trained on. The general public has no use for this explanation because they aren't the one deciding on the training data. So the only practical purpose is to offer some superficial causal mechanism that sounds like an explanation to that public, presented as an unchanging fact of nature, and distract from the equally true fact a biased model is only published or used iff it is trained on biased data and someone decides to go ahead and publish/use it nonetheless.

What does that mean? What is an AI researcher to do differently? Put their academic life on pause and wait for more unbiased data?

You're making a fallacy here, forcing the issue into black and white - if you mention first 'dataset bias', it must mean you 'offer some superficial causal mechanism that sounds like an explanation to that public'

It's not the message, it's the tone. There's a big difference between offering constructive criticism and being a condescending prick.

> You can disagree with that reasoning (although you'd be wrong). What you can't do, as a leader in AI, is to pretend to have never heard of it, or that it is so obviously wrong that it warrants no reply whatsoever.

There's a third option. You can just decide to not engage with the condescending prick at all.

We have one of those at work. His area of responsibility is a very cross functional one, and while it is important, he have a great deal of difficulty getting people to participate. Because participation usually means getting told you're an idiot. Loudly, forcefully, and with more words.

That seems to be the case here, only substituting "idiot" out for "racist".

https://en.wikipedia.org/wiki/Tone_policing

Yes, tone matters, regardless of your gender. A prerequisite for a mature adult conversation is for you to quit being hostile to me from the get-go.

Am I saying something controversial or micro-aggressive here? So be it.

Then, this is a confrontational tone too.

I have the distinct impression that what they want is for discussions to become personal. Don't let them get to you.

You shouldn't be tired of telling someone about a cutting edge subject that you are an " expert" of.

> "I'm so tired"

Tiredness is probably coming from trying to educate people on Twitter, you can't do that because tomorrow a fresh batch will come and you have to start over.

> "I'm so tired" bit

Yeah, it sort of implies that, yes, you're angry about something, but actually have no constructive advice or solutions, other than to accuse someone of being a bad person or even a racist.

The "I'm so tired" bit is similar to the "I'm not here to teach you" bit. Maybe someone can't teach you because they actually have nothing to teach.

No, "I'm so tired" means that it's an argument they have replied to before. She has written dozens of research papers, some of which probably contained arguments other than repetition of "you're a racist and a bad person".

You of course know about this, or would now about this considering she is called an "AI researcher at Google" in the title of this tread. So the only lazy and un-constructive criticism is to assume she has not made any arguments beyond your one-line caricature of what you believe her to be.

Then what is Gebru's solution to the FlickFaceHQ database having too many pictures of white people?

Expanding the dataset and/or using a subset of it?

Gebru would disagree:

> “I think that now a lot of people have understood that we need to have more diverse datasets, but unfortunately I felt like that’s kind of where the understanding has stopped. It’s like ‘let’s diversify our datasets. And that’s kind of ethics and fairness, right?’ But you can’t ignore social and structural problems.“

This seems to be saying that Yann LeCun should actually be addressing social and structural problems, rather that just applying neural networks to data sets.

https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...

IMHO she is saying it’s reasonable to believe that social and systemic issues have an impact on creating ML models (as the teams designing, building, and deploying said models are members of society) in ways in addition to the data sets.

I don’t know what the right answer is but plenty of institutions come up with ways they attempt to deal with systemic biases. For example, blind auditions for positions in orchestras (you only hear the person auditioning, you can’t see them). Used to not be a thing. Now it is.

Point being that when people are motivated they are able to come up with stuff.

Sure, but Yann LeCun is a neural network expert. Should it be his job to tackle social/systemic/institutional problems? What if he doesn't want to?

I think the point was reciting the (true!) 'biased datasets create biased models' argument at best avoids, and at worst actively undermines those bigger questions.

Someone in another thread on this had IMO a great comment[1] summarizing the sentiment:

> If you are going to say "Well, garbage-in, garbage-out!", why do you keep putting the racist garbage in?

[1]: https://news.ycombinator.com/item?id=25286446

It's really a stretch to call the Flickr-Faces-HQ Dataset "racist garbage."

It's a collection of images uploaded by random flickr users with permissive licenses.

https://github.com/NVlabs/ffhq-dataset

Yes it should if the networks he builds have an actual impact on real people because of their social/systemic/institutional status. If he doesn't want to do it, he shouldn't be working on networks that have this kind of impact.

What if you're a nuclear engineer that just wanna make the hot bit go hotter, and don't want to care about the nasty habit that radiation has of killing people?

Google “Gebru arxiv”.

If you’re confused as to whom Lecun is, google also “lecun arxiv”.

Honestly, I am not even sure _how_ they would fix the system(s) they complain most about? Assume Yann is wrong and Timnit is right, what is the proposal for fixing the bias that somehow is inherent in the system?

She says 'let us lead and you follow' -- sure...where? What are we concretely doing about the "in-built bias" problem?

The quality of the "argument" here is F--. Basically, it's, "nothing you say matters, only what we say matters." Phrases like "just listen" are essentially, "you will respect my authority!"

Yeah, that Lecun argument was a really bad first impression of her from me. I guess she found out life isn't Twitter and you can't just go around telling everyone they are bad and wrong without consequences.

She will land on her feet. There is a huge demand for ideologues willing to lobotomize ML.

“Let’s generate fake data with GANs to prevent our AI from discovering politically uncomfortable regularities!”

It’s ugly work but someone has to do it, right?

This is a rude dismissal of an entire field within machine learning and statistics (ethics, bias, and robustness).

Rude, maybe but not very. Dismissal, yes. The belief that AI should learn reality as it exists and not some ideologically purified version of it is a completely rational one, and dismissing the "work" of those so-called "ethicists" is no more problematic than dismissing the "work" of the Chinese censors who purify the Chinese internet. Indeed being dismissive of that work is to be praised, as it shouldn't exist at all.

Name the specific true facts about reality you think these models are capturing that are at risk of being 'purified' by AI ethicists. List them.

Well, read any AI bias paper. Facts about reality these models capture and which are considered problematic include stuff like "doctors are more often men and nurses are more often women" or "most people on the internet are white" or "most stuff on the internet is written in English". Basically the internet is mostly built by westerners and any facet of that gets classed as "bias" rather than being what it is: the truth.

Even trivial facts like in English "men chuckle, women giggle" has been used as examples of bias. That's not bias though, that's just how the English language works.

For example:

https://ai.googleblog.com/2020/04/a-scalable-approach-to-red...

[flagged]

What politically uncomfortable regularities are those? Please name the specific uncomfortable true facts about reality that you think these models are capturing.

"life isn't Twitter and you can't just go around telling everyone they are bad and wrong without consequences."

I wish this were true.

It is true, just not as often as we would like it to be. No “justice system” (informal here) punishes perfectly.

Jeff Dean, Head of Google AI has responded (original article has been updated.) claiming the following:

>Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resign from Google.

> This happened to me last year. I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google

Self entitlement or she forgot she was just an employee, and not some sort of independent tribal leader fighting google from the inside.

> I think if you threaten to sue your employer, the employer isn't too much at fault and just covering their own neck when you later threaten to resign and they take you up on that offer.

It's not clear to me here what you're advocating for -- a lawsuit wouldn't be raised by such an intelligent and virtuous individual unless there was some clear wrongdoing. Are you saying that you think workers shouldn't sue their employers if their employers are breaking the law?

in that case sounds like she deserved what she got

Just as an FYI, LeCun has not actually quit Twitter, he still posts stuff on there. For those curious I wrote up a more detailed summary of the episode (https://thegradient.pub/pulse-lessons/). TLDR, imho he was a bit unthoughtful in how he communicated with a more junior researcher, and got some respectful flack from it from various people, it was not that big a deal. But of course you can make up your own mind on that.

In any case, Timnit Gebru (this particular AI researcher) has been recognized as having done some pretty important work in AI ethics. Sounds like this whole thing is messy, but thought I'd mention that for more context.

Well, reading the thread, it's pretty clear that she was correct. There is a second step of metaoptimization which incentives the researchers to encode algorithms to produce algorithms that more easily bias along a certain way. And she is correct that benchmarking models against biased datasets can lead researchers to bias for architectures that are in turn more likely to have a certain bias.

I'm sure LeCun knows this too. It's a fact that has been known for almost 40 years now, that for a given generalization system multiple bias modes are are possible, and of course the process of optimizing architectures against a biased benchmark will change the learning bias of the model. It's not just datasets that are biased, generalization systems are necessarily biased too.

As for the last line, I don't know that it should be expected for employers to fire employees pursuing recourse against sexual harassment.

I'm sure we can agree that the tone wasn't good, but she does have a point in saying that it's not possible to reduce the bias to datasets used to train the final model.

The paper : http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf

TBH it doesn’t matter if she is right or wrong. Her tone and angry writing is pretty harsh.

The Twitter replies read a lot like her Google message thing. She’s got a lot of good things to say but needs to work on the delivery.

I think she’s playing the victim and race card a lot more than she should. She’s got all the ammo she needs to make her arguments without the angry victim tone.

I agree that her tone is not good, indeed. It's true that this is a big problem on Twitter especially. I still think it is an interesting and under-discussed point, though.

Have you considered that there are many other people with the same thing to say, who are saying it with the respectful tone you demand, and are being just as politely and quietly ignored?

Yes but I’m not a prominent AI Ethics Academic with ties to 2 of the best colleges in the country, Stanford and MIT. Who also had a very good job at one of the most sought after companies in the world, Google.

When she speaks people listen. Nobody listens to wil421.

I think the simplest thing to be said about messaging and tone is this: If your goal is to say things to convince people to change their behavior, it makes sense to spend some time thinking about how your message and its tone will be perceived, since that has a big impact on how people receive and process the message.

Based on a long career of watching people trying to convince the others, attacking people is typically a failing strategy. Instead, learn how to message in a way that is non confrontational, and most importantly, make it clear what is your opinion, versus fact.

Well, she didn't get ignored. Instead, she got fired. If that was her goal, then she succeeded. If it wasn't, maybe she should have taken a different approach.

Tone policing is a common way to silence people though.

Criticizing tone policing has to be self defeating.

Reply on news.ycombinator.com