We read the paper that forced Timnit Gebru out of Google

I have some experience with this. Not to nearly the same degree. But I had something of a tiff with an exec my second day on the job in a prior role.

Would you feel differently if, say, Apple hired a "Privacy Watchdog", with a long history of activism on the topic? Someone you could trust to speak out if something was amiss.

If the person is later fired, that's a sign that something is wrong at the company. But if they stay, and have generally good things to say, that's a sign the company can be trusted.

I do think this is a good system!

I am wholly uncomfortable with any discussion of politics in a workplace environment like a mailing list. I am more than fine with employees choosing to associate politically outside the workplace and workplace spaces, no matter how radical I find their views. This is in itself political - it supports the status quo - but anything else is inviting dissent and combativeness, and situations like these will keep happening.

That said, I've not seen the email outlining her conditions "or else", but I feel I'd have very much taken the same stance as you, given the surrounding coverage of what was in that email. Ultimatums to your employer don't often go well. And perhaps this is a good thing for her, because she may leave for a place that better suits her.

-

[1] my derisive use of this term does not stem from actual efforts at inclusiveness, those are good, but from surface-level attempts at it that end up feeling performative, at best.

To be fair, some people are aware of the people involved like Jeff Dean, but I look forward to a shift in the future where individuals within companies are as popular and held to the same standards as individuals within politics. Some of these corporate individuals already seem to have just as much power so hopefully it doesn't stand to be too much of a stretch in an ideological vision for the future.

With respect to this particular case of Timnit Gebru, it sounds like she was already on her way to being let go. From reading her Twitter, it seems like she has a flair for the dramatic which could make her critiques of people potentially come off as needlessly harsh and unconstructive. Whether that's good grounds to fire her may only be known to those within the company who interacted with her most I guess.

It doesn’t matter the topic that’s the outcome

Not to mention healthier for everyone involved

Google employees are way too comfortable with that message board they have over there

Abstract away from the particular goal here, the problem itself is difficult and interesting. How can the leader of a large organization change the organization’s culture?

Real change takes hard work and time, and patience, and also playing politics with powerful people you dont like.

Sounds like she basically made a one sided hit piece which was against the work some teams of brain, and took the "my way or the highway" approach. That wasn't so smart from her either...

Her manager, however, apparently would not have: https://m.facebook.com/story.php?story_fbid=3469738016467233...

Something is seriously wrong, BTW, when a manager has to find out from their own ex-employee that that employee has been fired by the manager’s manager, who, while not informing the manager, has sent a message about it to the fired employee’s direct reports.

Irrespective of the merits of the firing itself.

* Many jurisdictions impose strict controls on the use of racially biased methods (eg: redline laws like the fair housing act in the US).

* Different advertising companies have access to different targeting methods (eg facebook has your social graph, apple have your app use, google have your search history).

* Language models are a key technology for google to maintain advertising relevance & thus keep adwords competitive

"Models trained on modern language reflect societal biases" may seem like an obvious fact, but once google says it publicly they no longer have a basis to deny it in court.

I think this race has been ongoing for a while now

I think it's quite misleading to compare the energy usage of an industry-wide research effort to individual consumption. The graphs look bad - "wow, 626,000 lbs! that's 284 metric tons of CO2! a plane flight is way less!" - but there's a fundamental difference between "progress on a problem being worked on by thousands of highly-paid researchers" and "I bought a car".

Meanwhile, the worst power plants are generating on the order of 10+ million tons of CO2 every year. There are at least a dozen of these in the US alone. Car factories are emitting hundreds of thousands of tons of CO2 (Tesla is somewhere around 150,000 tons a year, apparently, and it's designed to be efficient). Perhaps activism around CO2 emissions in ML training might be better focused on improving the efficiency of those things instead, seeing as a 1% improvement would outweigh the entirety of the NLP model training industry. It's certainly good to keep in mind the energy costs of training in case things balloon out of control, but right now the costs relative to the results seem small and not worth highlighting as some forgotten sin.

It's not a fixed cost though, it's just how much was spent on this year's iteration of the model. The overall point being made is that model training costs are growing unbounded. Next year it could be 30 cars' worth or whatever for BERT-2, then 600 cars' worth for BERT-3 the year after. That's what it's warning against. At some point it isn't worth it.

Yes if you do run 240x p100s at literally 100% 24/7 for a year you get the power consumption of 5 cars. This run never happened though, this all ran on TPUs at lower precision, lower power consumption and much lower time to converge.

If anything this tells you that electronics are ridiculously green even when operating at 100%. I've never profiled world-wide carbon production but something tells me if you wanted to carbon optimise you'd be better served trying to take cars off the road and planes out of the sky.

In particular, it is being applied right now to rank Google search results, and probably responsible for lots and lots of Google's profit. You should be skeptical of Google's appraisal of the paper that is material to Google's profit.

Meh, both parrots and language models are inferior to humans in producing language, but one is more useful than the other. And real parrots are also stochastic, like all living things.

Gebru is the type of person who defines "bias" as anything that isn't sufficiently positive towards people who look like herself, not the usual definition of a deviation from reality as exists. Having encountered AI "fairness" and "bias" papers (words quoted because the words aren't used with their dictionary definitions), it's not even clear to me they should count as research at all, let alone be worth reading. They take as the starting premise that anything a model learns about the world that is politically incorrect is a bug, and go downhill from there.

This is historical revisionism, colored by your misremembering of the events. If you go back and look, you'll be unable to find Gebru encouraging anyone to attack LeCun.

Nor will you find LeCun making an effort to engage with Gebrus research. He never so much as acknowledges the existence of her publications when asked to read or comment on them.

E: as an amusing example of the importance of this kind of research, my phone decided that when I typed "Gebru", I meant "Henry".

To make an example that's en vogue right now, AI explainability. Nobody even has a definition of what it means for a model to be explainable (is a linear regression "more explainable" than ML? isn't Google search far less explainable than any model of anything ever?), but a reasonable framework for that concept could certainly be interesting.

Obviously, serious frameworks are done with definitions and math, not with words and storytelling, but all the air around those things seems to me more the fault of politicians, crap journalists and freaking idiots on social media (the current term is 'influencer', I reckon) rather than an issue with the field itself.

The problem that AI ethics research addresses are ethical problems that executives and employees aren't paying attention to. It may seem obvious when stated explicitly because of the amount of ease it takes to grasp the concepts (and the seemingly simple derivation of cause and effect relationships), but I assure you it is not obvious to a lot of people I know in the field at least.

There's also a clear misunderstanding of what ethics research should entail:

* AI being forced to chose between two "bad" scenarios will result in an unfavorable outcome for one party

This is a trivial result that doesn't hold much value as a standalone observation and probably wouldn't be touted as a research point in a respectable publication of AI ethics.

* AI could reveal truths people don't want to hear e.g. it might say the best team for a project is an all white male team between 25 - 30 rather than a more diverse team. It might say that a 'perfect' society needs a homogeneous population.

The fact that you made this comment may be a cause for an ethics discussion in itself. You used "truths" to describe the statement "the best team for a project is an all white male team between 25 - 30 rather than a more diverse team." This shows a disregard for the reality that most data is contextualized and biased. Using terms like "best team" and "more diverse team" make the statement like the one you made at risk for having took a misguided conclusion from data.

Maybe the following revised statement would be closer to what we can call a contextualized "truth" generated by an ML model:

"Teams that comprise of white males between 25 - 30 have a statistically larger chance of meeting milestones set by leadership rather than teams with one or more non-white male."

Even statements like that aren't complete as my definition of "meeting milestones" could be sourced from self-reported data (in which case it could mean that white males just self-report more milestone completions).

* AI could disrupt a lot of lower paid jobs first without governments having proper supports and retraining structures in place

A problem to consider, sure, but this is one of the more popular observations and has been echoed over time within the context of technological advancements in general.

Note, I’m not saying Google discriminates, but if they do then your statement doesn’t contribute to their defense.

(Are you confusing this with her email that said that participating in Google's feel-good diversity and inclusion work, specifically, was pointless and should stop, so that people could focus on their real jobs - such as research - instead of wasting time placating unknown managers?)

Anyway, my theory re your question is that Reddit and HN are usually pseudonymous, and so you have no idea whether the person saying something actually knows anything about Google or actually participates in the industry. The norm on Twitter is to use your real identity, and so comments from Googlers and ex-Googlers are boosted there. On Reddit and HN, good-sounding first-principles arguments are boosted. The two models are better at different kinds of discussions.

I heard buying Twitter bots is cheap.

Another obvious problem: she keeps being described as a highly respected AI researcher, a leader of the field etc. Er, no. I read a lot of AI papers. I don't respect Gebru or the sub-field of AI ethnics much at all. What's happening here is simple: criticising a black woman in tech is impossible to do under your own name in corporate America, as radical leftists will immediately set out as a group to destroy you. The only acceptable position is to praise her and claim she's a genius. On HN and Reddit you can see what people really think and it's very far from that.

This is one of the reasons I prefer HN to just about any other online forum.

Jumping in on a bandwagon to signal group membership in an in group can help brand-building. Going against the in group is potentially fatal to your brand. The risk/reward balance does not favor genuine discussion of a controversial topic.

On HN most comments are up/down voted based on their content. The post authorship usually matters little. There is little benefit for the author to announce support of a popular position and little risk for the author to express an unpopular one. So discussion here can be more about substance and less about signaling.

The James Damore episode is another example. It is essentially 100% equivalent to what Timnit has done in this chain of events, but in that case the “wokeness” bloc on Twitter called for Damore’s firing and railed against his perspective - virtually a complete double standard compared to how they view Timnit’s behavior.

By the mostly anonymous nature of Hacker News, and the removal of any type of social credit for patronage of different ideologues, the commentary here is able to be much more nuanced and try to really account for a much greater variety of facts or perspectives.

Because there are lots of FAANG engineers in here (i.e. working at Google, or with friends working at Google, or working at companies who do the same nasty AI stuff as Google) so them unconditionally supporting Gebru on this would be against their direct financial interests. It's as simple as that.

I’ve recently noticed that this commonly occurs ie multiple relevant threads that make up the top 50 of HN over say a 72 hour period, all related or rifts on a similar discussion.

Wondering if there is a way to group them as part of the same topic/submission? (thus saving you the manual work of posts like this). I appreciate this would (i) require a code change and (ii) would shift the HN model from individual threads based on 1 URL submission, but just thought I’d suggest it to the HN brains trust.

In other news: Keep up the good work that you do on HN to give this online community ‘structure’.

Google (probably) wanted to bury it because many of those clickbait headlines would have been negative to google.

The impact of that on AI and the difficulties in counteracting it are not obvious to anyone who has ever used the internet or even—from, among other bits of evidence, public clashes Gebru has had with people who work in AI outside of ethics—not even to everyone building and training AI models on public data that is impacted.

What we're seeing here is a simple problem spilling out into the public: a lot of AI research is being done at a small number of companies that are in turn dominated by a tiny minority of people with politically extreme views. Those people consider control of language and expression very important to achieve their goals of a racialist and gender-biased society. But they're building AIs which require training on huge language corpuses, and thus inherently learn the way people writing language actually think.

Meanwhile the techniques to do "mind control" of AI aren't that good, at least not yet, so this leads to a lot of tensions when AI learns something true but unacceptable to the woke mindset.

> [...] Though Bender asked us not to publish the paper itself because the authors didn’t want such an early draft circulating online, it gives some insight into the questions Gebru and her colleagues were raising about AI that might be causing Google concern.

So, we have on one hand, a researcher who got perilously close to litigation with her employer in the past (IMO because of missteps on both sides).

On the other hand, we have an employer that then was skittish about telling her that they didn't want the paper published (to protect the employer's business interests, mostly, it seems, while maintaining a veneer of open research organizations). And resorted to small statements through HR and intermediaries demanding retraction.

This relationship has broken down; there's no ready process to tidy up the misunderstandings.

I will say that Google's claims to be fostering an open discussion of AI ethics and confronting potentially uncomfortable truths on this path are looking a bit more dubious, though Ms. Gebru doesn't look so particularly easy to work with, either.

Models like BERT aren't just trained once either when they are developed, but trained again with different domains, different parameters, different tasks in some cases. There is also fine-tuning (more frequent, less carbon intendive), so these are real environmental problems, and others have pointed them out.

CO2 impact for stuff like this - we probably have bigger fish to fry.

If face recognition makes the old racist "they all look the same to me" declaration, then peaceful protestors get arrested for looking like criminals.

It can be used to (mis) identify them if the higher error rates are accepted. The problem isn't really the tech, but its application.

Like how, with activism, you can pressure the NYT into using jargon like “Latinx” when no one outside small progressive circles uses them.

What the AI should say is hard given that there is no one right answer to the question of what any individual should say. Different contexts change the equation completely. Seems like a nightmare to define the behavior or test it.

The right solution may involve training a model on everything we have access to and finetuning it based on the context you want to use the model in and the historical examples we will build up of mistakes previous models have made.

For example, here is Hispanic Congressman (D) Ruben Gallego on the subject: https://twitter.com/rubengallego/status/1324071039085670401?...

This sounds a lot like taking what Google said without including Timnit’s point of view. Is it really fair to disparage an article for having a title like that when you’ve completely ignored such a large part of the issue?

clicks. after a decade during which they lost money over fist not knowing how to monetise their product on the internet, they all went to the lowest common denominator: clicks. this in turn forced them to start bending the truth (something that tabloids where known for, and for which they got huge profits).

basically we still don’t have a viable business model for this industry. the only one that works needs headlines such as the one you mentioned to function.

the interesting bit is that publication such as Nikkei or the FT are still top-notch, but these are niches, not general audience publications. (also my FT subscription is £30/month and that’s a lot of money if you’re not in that niche)

Timnit says she was fired: https://twitter.com/timnitGebru/status/1334352694664957952

Jeff Dean's email says the article was "approved", though clearly not approved enough: https://www.platformer.news/p/the-withering-email-that-got-a...

Other researchers at Google say that this level of scrutiny is highly unusual: https://news.ycombinator.com/item?id=25307618

So in fact it is you that is twisting the narrative by asserting that the truth is publicly known, and that it is exactly as Google describes.

We all would love to see an evidence of it.

You might be interested to know that I’ve changed my mind. The reasons are documented here: https://news.ycombinator.com/item?id=25308233

Basically, every one of those arguments evaporates when you dig into the details. She was doing her job. The paper was anodyne and perhaps even boring. She was rightfully pissed off that some middle manager was telling her and her four coauthors that they must retract their paper (the reviewers later posted on Reddit that they would have been happy to accept edits). Jeff was micromanaging her citation list, which is almost unheard of, apparently. Multiple google employees came out of the woodwork to say “there is no such review process, other than a quick scan to see if you’re leaking company secrets by accident.”

All of this is fascinating. Because, as far as I can tell, the entire AI community is now on her side. The only people who keep bringing up her character flaws are people who don’t do AI / ML. Or at least, they seem to be pretty quiet now that Jeff revealed Google supposedly has some weird academic review process no one’s heard of till now.

Your feelings about the journalists are completely warranted, to underscore that point. But it’s odd how much momentum is building. Try to ignore the circus; you’ll be surprised to find there is substance behind the claims. I was as surprised as anyone.

And to reiterate, pick any one of the arguments you mention, and really dig into it deeply for details. Try to verify the claims. The most you’ll get is that she was curt. And I remind you that it’s usually known who the reviewers are at any given venue. Even if it’s blinded, you at least know the general group of people involved. In this case it was highly unusual to do some kind of anonymous, selective, ad-hoc enforcement of rules that really don’t seem to benefit the scientific process in any way.

I'm unsure whether "improvement" is the right synonym for "change" in this situation.

That’s false - she, at most, threatened to resign.

And often it is a damn that need only break once. Corporate ethics as a whole seems contradictory. The only ethical rules which a company may obey in perpetuity are those that do not for long conflict with profit and those which are backed by effective laws.

Even when done with a positive intent, I fear creating positions such as this is begging for eventual corruption.

The response was to decline to meet and fire her.

It’s entirely plausible that the paper is bunk; however, when someone is willing to come down that hard to prevent an idea being published, I tend to err on the side of “worth finding out what”.

While I don't really understand the emissions argument, the rest strikes me as very defensible. If the best language models need giant datasets to excel, it is very difficult to ensure your AI is trained on reasonable data.

I wouldn't want an AI therapist to be trained on 4chan. Now obviously nobody would be that stupid, but unfortunately it seems we are not far from it.

Unless we build models that rely on less data, it will be difficult to prevent problematic biases in the AI we put into production.

If that is the argument Timnit makes, I think that's the exact type of work I would expect from an AI ethics department. And good work at that.

I'm not blaming you solely but the recurring idea that it can is worrying.

Great point though

- Outside media continues to refer to her as Timnit including NYTimes which affects HN users

- If someone’s name is Gorot Trzebiatowski (please forgive me if you have such a name), most people would go with Gorot because it is shorter and easier to pronounce / write. Subconsciously, these things happen.

You’re right - No need to assume malice where probably there isn’t. This is conspiratorial thinking - find some pattern of data that supports mainstream narrative and make conclusions from it. This is what QAnon conspirators do all day.

Although I’m curious about your motivation for analyzing this :-)

Regarding first/last name confusion, how many people here on HN _know_ which is her first name and which is her last? It's a lot easier for Western Anglicized speakers to get it wrong with non-Western non-Anglicized names than with "Jeff Dean", whether intentionally or not.

(Now, I don't actually have enough of a sense of Ethiopian ethnicities to be confident as to whether this holds true for all of them and whether Timnit Gebru belongs to one for which it does, but the case of @DrTedros certainly left me with a general heuristic saying that referring to her as Gebru may be an ignorant foreigner move whereas using the "first" name (as US academics generally refer to each other anyway) is almost certainly safe.)

Maybe if you have an unpopular language, that's just unfortunate and please encourage your kids to learn English (which they probably do at school just about everywhere anyway) so you don't perpetuate the same problem to the next generation.

Man, people really do want to have their cake and eat it too. If you want to publish research freely join an academic research lab, if you value money join an industry lab. You can't have both of those things.

But the research is not even published yet

Google consumes a vast amount of energy: "10.6 terawatt hours in 2018, up from 2.86 terawatt hours in 2011."[1]

If training and retraining models is a significant and inefficient part of google's energy consumption, the point doesn't seem insignificant(edit: The most advanced AI model involve as much as computing and energy any programs ever created [2], btw). I'm biased by the impression Google's actual search results haven't improved very much but I don't think I'm alone in that impression.

[1] https://www.statista.com/statistics/788540/energy-consumptio...

[2] https://openai.com/blog/ai-and-compute/

Data centers around the world account only for a tiny fraction of electricity consumption or carbon emissions. Do you even account for the reduced car and air travel due to remote working and online shopping?

How are advanced few-shot learners like GPT-3 even remotely a problem, training less models is somehow worse? Do they even know what they are taking about?

Its all very confusing, but a lot of the work done by grievance studies becomes immediately easier to understand once you realize they are arguing in bad faith.

They lose nothing but face.

A workplace that isn't hostile to people of color and evidence of the cause of dismissal

If a company is responsible in how they manage, follows policies, etc they are fine. If executives or others are allowed to misbehave and the company is too cheap to buy silence, things may not be fine.

What’s the real story here? I don’t see evidence of incompetence. But you can be fired for any legal reason in absence of a contract. Maybe there’s some unknown political or other issue. Maybe some conduct crossed a line. Who knows.

The word "fired" often is reserved for terminating someone for cause, where you really broke the rules and get no severance, not even the two weeks minimum that is customary. Non-performers usually aren't treated this way. I'm not sure this is really what happened in this story.

Google has set up a research organization that ostensibly is empowered to ask and ultimately work to openly resolve difficult questions like these... but when the rubber hits the road, they instead throw the researcher under the bus.

I think it is going to be extremely hard. From the same article, I opened a tweet and look at what a high voted reply is: https://twitter.com/PocketNihilist/status/133495412981528985...

This line of reasoning means, you can't be pro diversity and fire someone from the underrepresented groups at the same time for their behaviour.

People are conveniently choosing to forget this is the same company which not long ago fired a person when he complained about the company being too pro diversity in their hiring.

I really hate Twitter and its mob culture.

For them to be one of 99.9% of the world who can't mobilize a following to create a complaint about this?

Are we creating a society that makes it impossible to fire a person of color

We so far from such a situation like that that your complaint is absurd. A few places with a history of discrimination may have trouble firing the few people of color they might hire. That's about it. In the real world, incompetent people get fired and often competent people as well. A few people may make a career of playing the race card but that's a limited number of people.

Both racism and opportunists "playing the race card" can be real at the same time.

WTF dude.

The core argument as to what you say is a "woke" interpretation of the world is to reshape the world in such a way that inherent privilege (for example being rich, being white, being male) does not completely predetermine anyone's outcomes. So that you can still "make it" even if you don't start out as rich, if you're not white, or male. Basically, if you want to boil it down to slogans, it's "the American Dream for everyone". Or, equal opportunities – not equal outcomes.

I'm not sure why that is a immediate threat to democracy, free speech, or your way of living.

But maybe you think that we still need to debate whether or not we need to hire more women?

And even if you argue that we do NOT need to take action now to bring more gender equality into tech - who is stopping you from making that argument?

Judging by all the comments on the HN threads it seems there are plenty of voices expressing opinions.

I assume that this chorus is mostly men - 86% perhaps?

Suffragettes battled hard to allow women to vote, for example. While reading your comment I wondered whether you would have considered their movement to be "woke" at the time.

For their part, Google likely truly does try hard to behave ethically because it tends to be good for their image and business.

But it may also be true that there are dangers and risks involved in AI research that Timnit (as an expert in the field) believes have the capacity to perpetuate inequality on a long timescale.

You state that a sense of urgency around that indicates an ideology. I'd suggest that almost everyone who participates actively in the kind of liberal democracy you defend requires some kind of ideology to guide their decision-making.

The most trustworthy people may adjust and refine their ideology when faced with contradicting facts and evidence, and to do so they may need to understand and reason about those counter-arguments.

There seems to be an underlying sense in some of these threads that the financial and technological setbacks that the tech industry might suffer as a result of adopting more ethical policies and listening to employee concerns wouldn't be worth the cost.

There is less discussion and optimism, for some reason, about what the benefits of a happier and more transparently equitable work environment would be.

Speaking of ideologies, I think this hints at a sense of company loyalty and perhaps national loyalty, with a possible fear that criticisms may be a form of subversion, accidental or malign.

Those loyalties and suspicions may help the participating groups succeed, or could equally lead to their failure if the surrounding environment changes. That is, perhaps, the market at work.

The point is, I'm not a Google shill.

Still, on this case, on a factual level, the only real dispute is whether this exchange:

"Do X or I quit" "Ok, your final paycheck is in the mail and IT will be in touch to organise equipment returns, effective now"

Is "accepting a resignation" or "firing". Neither side is disputing that this is how it went down.

On an ethical level, again, I'm no fan of Google but Timnit Cebru's previous public actions don't paint her in a good light while Jeff Dean's doesn't have any notable enough to sway my opinion one way or another on his ethical trustworthiness.

So based on that, I (and many others) do end up siding with Google. is that a pro-Google push? is someone co-ordinating this? If they are, they haven't contacted me. Don't mistake the fact that Google is often unpopular here with the idea that no Google action can be supported here without interference

No, I haven’t lost respect.

Without taking sides (I don't know the full story to understand who is objectively in the wrong here), that's not completely true.

She basically gave an ultimatum - meet my demands or I can work on a last date.[1] It is quite common at big tech that once you resign - officially or unofficially - you could be asked to leave effective immediately. Usually happens when you work in critical areas or moving to a competitor or a disgruntled employee.

Could this have been better handled? Maybe, but no matter who was in the right or wrong, she wasn't technically fired.

[1] https://twitter.com/timnitGebru/status/1334343577044979712

Such is the product of a country which hasn't had any real problems in half a century.