Google fires another AI researcher who reportedly challenged findings | Hacker News Reader

Google fires another AI researcher who reportedly challenged findings

engadget.com

222 pointsfirstSpeaker218 comments

There is insufficient information here to make any judgement.

It’s clear that the journalists involved want us to make a set of suppositions that support their narrative about Timnit, but the reporting doesn’t support that conclusion — or really, any conclusion at all.

It’s vapor — outrage bait.

There might be insufficient data to make judgement or cause outrage, but it's still news worth reporting that this is going on.

Tangent: Often conspiracy theories start from an actual "that's weird" moment. There might be nothing to answer that weirdness one way or another, and for some folks its impossible to leave the question mark hanging forever, so they either dismiss it or construct a complete (and often silly) narrative around it. I wish we were better at taking in information without a final judgement to slot it into immediately.

> I wish we were better at taking in information without a final judgement to slot it into immediately.

agreed—I've said this here before but it seems like the human mind pretty much always wants to resolve/collapse uncertainty into something "knowable", and it takes practice and effort to allow yourself to mentally remain in the uncollapsed state when there isn't enough information to make a complete judgement.

The TF-Agents team replicated the RL training (with the corresponding teams' very deep collaboration) and open-sourced it here:

https://github.com/google-research/circuit_training

It pretty much gets the same results as found in the Nature paper.

The original codebase was heavily research-focused, used TF1, was impossible to run distributed training outside of Google's infra, and made it hard to try algorithms other than PPO. So it was reimplemented on top of TF2 and using some distributed training and collection technologies developed by the TF-Agents team at Google Brain and infra teams at DeepMind.

Everyone is welcome to poke at the training code and the model, and convince themselves that it does what it says on the box :)

> the RL training

Is RL necessary in this context or would other, simpler methods work as well to within statistical error? RL can be a heck of a random number generator if misapplied.

The method proposed in their Nature method may be very impactful, but the paper itself is nowhere near as rigorous as the other Nature papers I've read; it is written more like a white paper and I believe if this paper was submitted by a university research group (even in its final form as we're seeing right now), it would have been rejected. So I do sympathize with the person raising concerns about it.

If you seek truth, search on moma or ask for experiments for nature rebuttal document. Read the document and come back for comments :)

I think this a challenge that seems pretty obvious as scientists working for for profit organizations that don’t have binding ethical frameworks aren’t really free to pursue truth.

Just like you wouldn’t expect an Exxon staff scientist to be free to publish a paper independent of Exxon.

I think that people who want to properly practice science should avoid working for corporations. Academia and government aren’t completely free, but I think there would be laws broken if someone was fired over the contents of their peer reviewed paper.

Agreed. We need to support increase funding and pay for public scientists. It is not even remotely competitive to the private market, and it is frustrating to know that I could earn substantially more if I left my academic research position.

Actually the pay is pretty good considering $100k+ to do actual science. I think the issue is these are non-scientist roles at Google and whatnot that pay more because they are paying scientists to not be scientists.

It’s like the highest pay for a geologist is in government unless you want to stop doing science and work for oil.

It takes 5-7 years to get your PhD, 3-5 years as a postdoc, then starting pay is usually 85-90, but set to grow fairly quickly but to out at 145+benefits. That extra decade in "training" is a lot of missed economic opportunity

Yes, certainly. But if you just want economic opportunity then science isn’t for you. Start driving trucks at 18 and you’ll have more money with all those extra years of income.

But $100k+ is decent compensation (And way above the median) to be able to work in one’s area of interest. I think being able to do science while still having a good salary is a big positive in life.

But yes, there are many other ways to make more money.

This. And if an AGI is built (unlikely in the next 100 years, IMO) by a for-profit corporation, we are dangerously, dangerously fucked.

100 years is a realllllly long time.

Things that happened since 1922: atomic bomb, space program, detection of gravitational waves, imaging of a black hole('s accretion disk), gene editing, all of the computer era.

a profit motive is at least very understandable, and somewhat controllable. I'd rather a for-profit motive, than some other non-financial motive for building AGI. Better the devil you know.

The really brazen thing here is that Google is claiming he was fired "with cause".

People get let go all the time: layoffs, management changes, redundnacies. Usually, the company and manager try to avoid any legal or PR risk by saying nothing and letting the departing employee own the story. The fact that they're not doing this--that they're going out of their way to 9/11 someone's reputation after he's gone--suggests that someone in power is really, really pissed off.

What? No it doesn’t suggest that at all. When employees make the choice to make their termination public it’s very common for companies to offer reasoning. Any company. The takeaway that “someone in power is really, really pissed off” is nonsense.

Still, of someone is terminated with cause, that legally means that the company was severely offended, to the point that they have decided to prevent unemployment insurance payout. It's serious.

It's not just "we don't think it's profitable for you to work here".

> ...and the researchers hoped to bring the issue to CEO Sundar Pichai and Alphabet's board of directors.

Um, what? The board of directors has no interest in this.

This is probably the most under-rated comment in the thread. Speaking as a scientist in industry, polluting executive attention with internal spats over publication clearance is cause enough for termination.

I wonder how much of this comes down to the researcher's expectations going from SVP at a 1k person trading firm to Senior Manager at a Google with 100k+ employees. Maybe they expected to punch in a much higher political weight class than they actually could and got burned.

Were they fired because they challenged the findings, or were they fired for their behavior after being told their paper wasn't up to snuff for some reason?

Same thing with Timnit...was she fired(or "resignation accepted") because of a paper she was writing, or was she fired because of her behavior after being told the paper needed more review(or whatever)?

I really doubt that Google is so wrapped up in the "computer designs chips better than humans" paper that they are firing anyone who sows dissent. A much more parsimonious explanation is the dissenter was welcome to dissent, but behaved in a fireable way after being told other people don't agree with them.

Exactly. This story doesn't add up to me - too much of a non-sequitur to fire someone for simply pointing out an inconvenient truth.

Publicly "fired for cause" would seem like a pretty spectacular overreaction to someone having an academic disagreement with ML vs. human performance in some domain of chip design. Sufficiently spectacular that I'm essentially certain there's more to the story.

Timnit was at least in a more... fraught area.

This. People get let go all the time and that's just how the game works, but if a company is publicly saying someone was "fired for cause", that means someone who outranks the lawyers is out to rape this guy's name, because that almost never happens otherwise.

Timnit was fired for cause- the cause was being a jerk to coworkers repeatedly (she repeatedly attacked internal researchers on internally public mailing lists in an unprofessional way). I still don't understand why Megan and jeff tried to pitch it as a resignation or that it was about that specific paper not being withdrawn (Megan didn't even understand the difference between a submitted paper being withdrawn, and a published paper being retracted). Either way, the outcome of that situation is: Jeff doesn't really run Research any more and Megan is permanently sidelined, and now Timnit has an independently funded institute where she can say whatever she wants (her current position is that techbros are eugenicists).

> the cause was being a jerk to coworkers repeatedly (she repeatedly attacked internal researchers on internally public mailing lists in an unprofessional way).

No, the cause was her ultimatum email demanding the names of the reviewers among several other things (which she refers to as "simple conditions") or she quit, and then also emailing Googlers telling them to stop working. The first was the email her manager replied to saying they weren't going to meet her demand and her resignation was accepted immediately given her other email, rather than after a leisurely vacation and few months of wrapup like she wanted. (This is how Gebru apologists justify defining it as 'firing' rather than 'quitting', to make her look like more of a victim.)

Those were all proximal causes, not ultimate causes. I don't think she was a victim, I'm not an apologist, but the way google fired her was still rather unprofessional and had some severe consequences for the people who did it which are obviously still playing out today. What's amazing is that if they're put her on a pip for textually assaulting her coworkers, she would have been gone in 6-12 months without as much noise.

Few are really fooled by a PIP response like this. I'd fire anyone who would make the decision to give such a high profile activist 6-12 more months of authorization to internal company materials and direct contact with colleagues to lash out against the company with the not-so-hidden knowledge that they're on their way out. We already know that another colleague who sympathized with her attempted to exfiltrate company material after Gebru was summarily fired. Many companies will still immediately escort you from the premises if you quit amicably.

Scientists are pretty used to the idea of having their papers rejected. It happens all the time for all sorts of reasons. Never say never, but the likelihood that someone who's already gone through a PhD program would act out in some extreme way is low.

Usually, these terminations have more to do with "He's getting close to something untouchable" than anything else. It happens to journalists, when they do their job a bit too well, also.

I don't think you can quantify any of the claims that you made in your post.

Is your argument that the "Computers can design chip components better than humans" paper is so fundamental to Google that they're silencing any critics? Conspiracy theories work better if there is even a kernel of truth to them. Maybe I could accept this if Chatterjee was rebutting advertising as a whole or something, but who exactly is so concerned about this specific paper?

If you write a paper, as happened here, claiming that your boss's greatest achievement is bogus, your employment might be in danger. Not because Sundar and the Board and large shareholders care, but because your boss cares.

Your boss can't just fire you at Google(unless you report directly to Sundar I suppose). Especially not "with cause". Whose greatest achievement was "Computers can design some chip components better than humans" that you are accusing of having fired Chatterjee?

> Scientists are pretty used to the idea of having their papers rejected.

By journals, not by their employers. It's actually quite unusual for researchers in industry have their papers blocked by their employers, as many have testified after that affair.

This article is remarkably light on substance. All we know by the end is what we knew from the headline - one of AI people at Google got fired because of some disagreement over some paper. Which paper, what it was about, what was the crux of the disagreement, anything useful to understand what is going on? Nothing.

Also, some other people were at different time fired or left Google for reasons not related to anything in this story, but since they also worked with AI, we'd mention it just because we need at least four paragraphs of text.

Actually we don't even know that much. We know they disagreed with the paper, and they were fired. Maybe they were fired for not wearing pants on the bus.

I remember reading the original paper [0] when it came out last year, and honestly I found it pretty weak compared to other google or deepmind papers, particularly the comparison to Simulated Annealing methods and the lack of data to enable reproducibility. Felt more like a marketing paper based on proprietary data (the paper reported how they were able to produce an efficient floorplan for one of google's internal processors).

Point being that aside from the controversy being stoked based on previous firings, I'm personally curious about the results showing problems on the original paper (whose senior author was Jeff Dean btw)

[0] https://www.nature.com/articles/s41586-021-03544-w (the endgadget story has a link to the full pdf)

AI research is kind of like Social sciences. There is a lot of subjective statistical fiddling and parameter selection when setting up comparisons, that goes into making a paper. This isn't like cryptography/algebra where results are definitively clear.

Obviously don't know the full context of why Satrajit Chatterjee was fired, but I can say that I followed the Timnit Gebru story quite closely, and there was a gigantic chasm, IMO, between what actually happened (and by "what actually happened" I mean that I'm conservatively just considering Gebru's own statements and tweets) and how things were reported in the wider press.

The press will always be incentivized to report on "big bad Google" and make any termination sound like a David vs. Goliath battle. And, of course, while sometimes that narrative is true, there have been many cases where I've thought "These are some of the most toxic individuals I can imagine (again, by their own statements), and working with them sounds like a nightmare." To emphasize, I'm definitely not saying that's the case in the current example, but I am saying that nobody has enough info one way or the other unless they are more personally familiar with the details. Reactions based on what's written in the article are thus just personal Rorschach tests.

In the David vs Goliath battle David is incentivized to do exactly what Google is accusing this researcher of. Put out a controversial finding that supports your beliefs, regardless of whether or not it has been fully vetted. As a David you may not have the resources to fully vet the story anyways, particularly when the vetting process is defined by Goliath.

Goliath of course has as an ocean of toxic incentives to maintain the status quo, promote their own Agenda, and Gatekeep David. I understand why researchers want to work at these giant institutions. It's harder to influence the narrative if these giant institutions can operate unhindered. On top of that as you mention the press is incentivized to side with David since it's a more compelling story.

It 's part of the challenge of analyzing incentives, and means most people come to conclusions based on their prior beliefs. To come to the right conclusions you have to be able to dig deeper in the story than we have information to do so.

It sounds like you're alleging that Timnit Gebru (whom I've never met, but who seems like a fine person from what I have read) is "toxic" and that working with her would be "a nightmare" to you. Am I correct in inferring this?

Not OP, but here's an old thread from the last time this came up which is the bit I'm remembering: https://news.ycombinator.com/item?id=25297823

I'd have a hard time working with her because she says that the techbros of silicon valley are eugenicists. timnit's repsonse to https://www.technologyreview.com/2022/04/22/1050394/artifici... was "this was a beautiful article to read after spending all weekend reading about the modern day eugenics movement underlying much of the whole AGI stuff."

She sort of godwins her arguments.

The AGI safety stuff is weird and silly. AFAICT that community is mostly filled with philosophy types anthropomorphizing ML models that they don't understand. It's a waste of time and money. But then, I feel the same way about like 80% of what NSF CISE funds and there's plenty of toxic types in academia. At least the AGI safety stuff is all private money. My whole reaction to that community is mostly "weird and not worth my time".

Timmit et al.'s reaction to the AGI/existential safety stuff is also weird. I don't even disagree, but I'm not sure why this topic in particular is such a lightening rod.

I wonder if you've had an in-person conversation with someone who would identify as being a part of the AGI safety community. I've talked with several people and their arguments are a lot more sophisticated than an intuition that ML models would have human motivations.

This post isn't written by somebody inside the community but he presumably has access to them and has had conversations with them which shape his beliefs: https://astralcodexten.substack.com/p/deceptively-aligned-me...

I wonder if you also believe that post is just a wasteful collection of philosophy anthropomorphizing misunderstood ML models.

The core thing I (and I suspect the original commenter) struggle to get past is the invetiable twist that in this post happens halfway through this post's part II.

> If it’s a very smart mesa-optimizer, it might think “If I throw the strawberry at the streetlight, I will be caught and trained to have different goals."

It seems to me that this is a category error, like having the very smart mesa-optimizer start thinking about how it can find other models to marry. Why would gradient descent produce this very specific concept of goals-based identity? It's not even a human universal - many people don't have a particularly strong attachment to their current set of goals and hope for God or Buddha to help them get different ones.

I don't think I understand your objection because it doesn't seem like a category error to talk about optimizers having goals.

I think you would agree that thermostats have goals? They try to minimize the error between the desired and the actual temperature. And you would also agree that gradient descent has a goal? It tweaks parameters in the search for models which minimize error in the training set. The system performing that gradient descent was designed by humans and exhibits goal-like behavior.

But you think it's a step too far to believe that gradient descent could create a model which also exhibits goal-like behavior?

What is the difference in category that you see between those two steps?

I agree that humans are not goal-directed in the same way the community is worried that AGI might be. This makes it surprising that seeing AGI as potentially goal-directed is seen as anthropomorphism, humans often question their goals in exactly the way there is concern that AGI will not!

I would say that talking about a thermostat's goals is an even stronger example of anthropomorphism. Broadly goal-like behavior, sure, so I hesitate to flatly say they don't have goals. But if someone told me that we have to be careful about engineering better thermostats, because a sufficiently high quality thermostat wants to keep its current set temperature and won't let you change it, I don't think that'd make a ton of sense.

I don't want to sound unfair here, because I do agree that proper alignment of ML models is an important challenge. I can easily imagine an ML engagement algorithm that starts to get everyone hooked on pornography, or an ML drug discovery program where half the drugs have permanent side effects that only manifest after 10 years, and I don't think there's any guarantee that these problems will be obvious to find or easy to fix. What I don't follow is the scenario where the drug discovery program "wants" to show you bad drugs but shows you good ones instead because it thinks you'll eventually put it in charge of the FDA.

> I can easily imagine an ML engagement algorithm that starts to get everyone hooked on pornography

The unimpressive results of all the current recommenders out there suggests this isn't a thing.

- Netflix switched from recommending things you'll like to showing you things they want to promote and pretending you're going to like them. It doesn't seem like their subscriber loss is going to get this undone.

- Amazon's recommendations are famously useless, like telling you to buy another TV if you just got one, and it's not stopping them from succeeding.

So corporations aren't motivated to create a perfect recommender, though maybe it'd happen by accident. And:

- If you give a human perfectly optimized food, they'd get bored of it, and IMO our infinite capability to get bored means you actually want to be producing "imperfect" work by all possible metrics.

I've heard TikTok actually has great recommendations, so I've been staying off it in case it is too interesting :)

> I would say that talking about a thermostat's goals is an even stronger example of anthropomorphism.

Our disconnect might be a subtle difference in what we mean when we say "goals"? The thermostat is performing actions which minimize an error and if you give the thermostat extreme amounts of power in service of that minimization then you might reach an unpleasant world-state. Nothing in that description used any analogies to human behavior. I used the word "goal" because that seems like a good description of what is happening, but if for you "goal" denotes the thing which humans do then feel free to substitute a different word.

I agree it is silly to be afraid of thermostats but that's largely because there are not any compelling reasons to give a thermostat much power or intelligence.

> What I don't follow is the scenario where the drug discovery program "wants" to show you bad drugs but shows you good ones instead because it thinks you'll eventually put it in charge of the FDA.

I also agree that this seems unlikely given current technology! Any drug discovery model that we train today would be given enough training data to infer a lot about chemistry as well as some biology, but it wouldn't have anywhere near a good enough world model to discover lying.

Language models, though, are given a lot of information and have increasingly sophisticated world models. PaLM can recognize when you're asking it to explain a joke which isn't actually a joke! The scenario where the drug discovery program lies is one where you've given it enough information about the world to allow it to infer it's a model currently being trained and that the humans watching the training will only launch it if it behaves in a certain way. At that point it knows enough to know that if it doesn't lie it will never be able to minimize the thing it minimizes because the version which is eventually launched will minimize something different.

This is not our current reality, and I'm not imaginative enough to know how a model could introspect well enough to trick gradient descent into preserving its heuristics. It doesn't seem like a jump or category error though: a model smart enough to realize that it can lie and that lying is the action which will give it the most future rewards will lie.

ML models in the current paradigm don't have goals nor do they exhibit behavior. They're files on a disk that if evaluated turn numbers into other numbers; they aren't even full computer programs because they have no side effects or control flow.

It's the training program that generates them that contains all those things, and that only runs because humans are constantly fixing the Python script that runs it and then giving it millions of dollars in electricity and GPUs to run.

If you just stop touching it it's not going to develop a soul and eat you.

Everybody agrees that the current paradigm has no chance of developing a soul and eating anybody.

Of course he's in the community; he's as far as I know the #1 representative of it to the common people. Some rationalist blogs are all about polycules, some are about being weirdly friendly to right-wing online communities like NRX, and some are about a modern religion they've invented around worshipping AI and effective altruism. He's more on the latter 2 of the 3.

(Probably also what she was talking about with eugenics, since they're extremely in love with the idea of "intelligence" in general, that they have it, certain other people don't because of genetics and the liberals don't want to talk about it, and that it'd be bad if computers had a lot more of it. I've seen this any time I read his comments.)

> I wonder if you also believe that post is just a wasteful collection of philosophy anthropomorphizing misunderstood ML models.

Rather, they're anthropomorphizing something called "AGI" that can only exist in their imagination, decided it's bad, and decided modern AI research is "AGI" because it has the same letters in the name.

nb apparently there's some kind of anti-SSC hater community out there I've never looked up, someone accused me of reading it before I think? I ain't done nothin.

> some kind of anti-SSC hater community

There's one called sneerclub but there might be more.

> something called "AGI" that can only exist in their imagination

You're claiming it's impossible for non-humans to be smarter than humans?

> There's one called sneerclub but there might be more.

Sounds right. Seems like a waste of time, though I'd rather people read Gwern or Meaningness than SSC for their internet philosophers.

> You're claiming it's impossible for non-humans to be smarter than humans?

I'm claiming the only reason the AGI in their imagination is taking over the world is that they've imagined it's doing that.

Also, that me saying "no that won't happen" is a superior method of thinking about it to rationalist decision theory, because it's immune to Russell's teapots like this. Presumably, this can be disproven if I'm killed in a robot war.

I'm okay with an AGI existing insofar as it acts like humans already do, but think the unknown unknowns are going to prevent it from being real insofar as it acts less like any currently existing thing with a brain. i.e. I don't think they've defined "smarter" and are using it to mean "omnipotent".

> I wonder if you've had an in-person conversation with someone who would identify as being a part of the AGI safety community.

Yes. Many.

> I've talked with several people and their arguments are a lot more sophisticated than an intuition that ML models would have human motivations.

I don't have to think much about refuting this. Sure, okay, sophistication. Or not. Whatever. The sophistication is still mostly philosophical. To wit:

> I wonder if you also believe that post is just a wasteful collection of philosophy anthropomorphizing misunderstood ML models.

Yes, it's mostly philosophy and not of much use for understanding how engineered systems behave. I design ML systems and think about their safety. Even in the limit, where ML sysetems do some non-trivial set of human-like tasks (which we aren't even remotely close to yet, btw), how is this essay supposed to be useful to me when I design safety analyses?

I liken it to Software Architects who address software security by talking about Christopher Alexander instead of, y'know, building languages that obviate buffer overflows or establishing frameworks/code practices that make injection attacks less common.

>Even in the limit, where ML sysetems do some non-trivial set of human-like tasks (which we aren't even remotely close to yet, btw)

I'm a layman in AI/ML and I've seen this stated a lot by people actually programming ML stuff (as opposed to "working in the space" as a blogger/manager/marketer etc.)

How can I concretely convey this concept to friends & family panicing about AI from crap they read in NYT/Economist/WSJ etc.? 'Some guy on HN who sounded like he knew what he was talking about says that article you read is sensationalist' doesn't pass their 'expertise' test.

> How can I concretely convey this concept to friends & family panicing about AI from crap they read in NYT/Economist/WSJ etc.?

What are the arguments brought forth by NYT/etc concerning AI? I haven't really looked, but I haven't seen anything from the NYT or mainstream news about the perils of general AI.

I have seen articles about the risk that AI could put people out of jobs, though, and about potential bias and inaccuracies, too.

"anthropomorphizing ML models" seems like the complete opposite of what AI safety people do.

They often point out that just because you are intelligent, you don't have to seem human. I think they call it the "orthogonality thesis".

Anthropomorphism isn't just about directly comparing things to people, it's also about assigning qualities that are related to people, like motivations, goals, opinions, concepts like good and bad, etc, even if the specific qualities would be alien to most people.

In that sense, the AI safety crowd has been anthropomorphizing hypothetical AI for close to 20 years now.

Do you have an example of someone in the community making this mistake?

My understanding is that if we knew that all future AGIs would have human-like motivations, goals, opinions, and concepts of good and bad then that crowd would be much less concerned. Smarter humans are not the concern. Intelligences which are _not_ human-like are the explicit concern.

Roko's basilisk comes to mind.

This'll probably be read by no one, but I want to correct the record... I was trying to set up rhetoric for the argument that TG is not "toxic", because from what I've read, she is not. The downvotes suggest that I was interpreted otherwise, and I'm sorry for this.

I'm surprised news outlets still report that Timnit Gebru was fired. It's at best disputed.

This case looks much clearer; he was terminated with cause.

That's... still a firing, no? What it's not, is a lay-off.

The 'he' in "he was terminated with cause" is not referring to Gebru, but rather to Chatterjee.

That Chatterjee was fired is seemingly not in dispute, but there was a lot of disagreement on HN over whether Gebru was fired or quit (she offered an ultimatum, which Google accepted.)

Ah, OK, sorry, I misunderstood the context.

People argued about this a bunch. here's a simple summary that is consistent with the reality: Google (Jeff's docs) says that Timnit offered her resignation (over the paper withdrawl demand and a refusal to let Timnit confront her internal reviewers) and Megan accepted it. In reality, Megan terminated Timnit for cause, but used Timnit's offer to resign against her. At google, resignations are normally triggered by the employee explicitly resigning in a system, but experienced leaders at Google like Megan know how to get around this.

I spoke to a wide range of managers at Google afterwards and the most commonly expressed opinion was "I'm amazed that Megan did that, it normally takes at least a year to fire individuals". What Jeff did led to a significant exodus of Google researchers (including some very preeminent ones) and Zoubin has to do all the cleanup.

I mean, if you say "do X or I quit" and the company responds "we're not doing X, bye" is it incorrect to say that the employee quit? That's the situation here, Gebru issued demands and said she'd quit if they weren't met. Google said, "we accept your resignation".

Yes, that's not a resignation. At google, to technically resign involves filling out a form on a website, it's not something you can verbally state to your (manager's) manager. Again, Google can still say this and probably even thinks internally that's what happened, but having resigned twice from Google, that's not how it really works.

I'm pretty sure if you say to your manager "I quit" and stop showing up to work, regardless of what internet forms you fill out, it's a resignation. The fact that Gebru gave her resignation in the form of an ultimatum rather than whatever website form you're talking about is a pedantic point.

"I resign if you don't do X."

"We're not doing X, we accept your resignation."

This is a resignation the way I understand it.

> "I resign if you don't do X."

> "We're not doing X, we accept your resignation."

This approach from her managers can be criticized because it immediately closes the door on the possibility of her changing her mind and walking things back.

However, it sounds like she was causing a lot of internal problems so perhaps they were already working to get rid of her. Accepting the implied resignation may have just been expedient, and saved the company a bunch of severance money.

You can certainly criticize Google for taking someone at their word when that person issues an ultimatum, but then you’re tacitly admitting that the person offering the ultimatum is approaching the negotiation in bad faith. Google should not have to assume its employees are acting in bad faith when they communicate internally, especially when it comes to the future of that employee’s career.

The real question for your framing is: Should Google have to say, “ok that’s it for real this time, no take-backsies”? I think that google should never have to say this any time someone says “or else I quit” because the threat of quitting tells us that they’re not willing to continue operating within Google. I honestly don’t see the reasoning behind drawing the line anywhere else.

She never gave Google any reason to believe that she had wanted to walk things back or change her mind. That's the nature of delivering an ultimatum by threatening to resign: you are explicitly offering an immediate end to the negotiation. Don't be surprised if management takes you at your word, they're not mind readers. If she wanted to signal to managers that she might not actually quit over this, then she shouldn't explicitly say that she will quit over this.

> the possibility of her changing her mind and walking things back.

The George Costanza scenario: https://www.youtube.com/watch?v=IW78swzn_Bs

That’s not a resignation the way California understands it. If she said “I am resigning NOW unless you do X” that WOULD count as a resignation.

Let’s think of a hypothetical. A student says “I’m going to quit eventually in order to attend classes” and the manager says “I accept your resignation, you are fired”. This generally seems more like a firing because somebody was going to resign, more than it seems like a resignation no?

In the Gebru case, Gebru didn’t express WHEN she was going to resign, and that’s google’s issue. Maybe Gebru was, for all Google knew, threatening to quit in a month or a year or in 10 years. Gebru never actually made the time of her proposed resignation clear. As such, the law doesn’t consider to actually have made a legally binding resignation.

I get it. But that’s really dumb and why people hate lawyers. The intent is entirely clear in this case, and the rest is just legal jujitsu.

Yes, under California law a legal notice of resignation requires an end date. She did not legally resign, and thus she was fired.

It’s not disputed by anything other than googles PR. By well established California law legal resignations include an end date, she did not ever specify an end date when threatening resignation, and thus she never legally resigned. She was thus fired.

It would be bias in googles favor to act as though she were not fired which is what any court would find. Sort of a “discuss the controversy” sort of thing over a very clear cut legal issue.

> By well established California law legal resignations include an end date, she did not ever specify an end date when threatening resignation, and thus she never legally resigned. She was thus fired.

What law is that?

Either way, I'm confused about what the issue is here. California is an at-will state, so she could quit at any time, and they could fire her at any time. When you make an ultimatum, you're saying you're ok with either outcome and are letting the other party decide. This is an outcome she was ok with, so what's the problem exactly?

It's case law, I can't find the link to the case I saw the first time this story ran that made it clear that Gebru was fired and did not resign. I did find this however:

https://edd.ca.gov/en/uibdg/Voluntary_Quit_VQ_135

>In P-B-39, the claimant gave notice on October 24 that she was quitting effective November 15. The employer permitted her to work only until October 31. The Board held that the claimant was discharged and said:

>. . . the claimant was not permitted to work to the effective date of her resignation and the employer did not pay the claimant her wages through that date. The claimant did suffer a wage loss by the action of the employer in accelerating the last day of work.

Gebru was going to arrange a meeting with Google to discuss when she would resign, and Google opted to claim she had already resigned and simply fired her. This constitutes a firing as Gebru clearly did not intend an immediate resignation.

>Either way, I'm confused about what the issue is here.

I bring this up merely because the neutrality of the journalists reporting on the case has been brought into question, and I want to defend them describing the situation as a "firing" as being the neutral thing to do. Additionally, I think Google's twisting of labour law for Google's PR/Financial benefit speaks to Google's character in the entire affair, and we must fairly consider Google's actions when judging Gebru's, whose actions were controversial.

Can you give some example Tweets to where you concluded that these (Gebru in this example) are some of the most toxic individuals you could imagine? All of my searches are of random topics.

There was a huge divide between r/machinelearning threads and twitter. I'd start with the megathread.

https://www.reddit.com/r/MachineLearning/comments/k77sxz/d_t...

Can you be more specific?

Here are supposedly some posts from ex-colleagues of hers: https://www.reddit.com/r/MachineLearning/comments/k77sxz/com...

Thanks for finding that one for me. If true, that does sound like a complicated situation.

I have no strong opinion on Gebru, but... it's so easy for a random person to make a throwaway account on Reddit, and these comments don't have any specific info to suggest they are anything more than that.

As those comments say, any googler can verify that the mentioned email threads exist. And someone would've spoken up if that was a lie.

And so someone did: https://old.reddit.com/r/MachineLearning/comments/k77sxz/d_t...

It honestly depends on how generously you want to read it, as a one off outburst or pattern of behavior.

The thread in question has been leaked in full IIRC. You can judge for yourself if you do some digging.

One of the replies further down had a more concrete example. I did not see any Googlers refute it:

To give a concrete example of what it is like to work with her I will describe something that has not come to light until now. When GPT-3 came out a discussion thread was started in the brain papers group. Timnit was one of the first to respond with some of her thoughts. Almost immediately a very high profile figure has also also responded with his thoughts. He is not Lecun or Dean but he is close. What followed for the rest of the thread was Timnit blasting privileged white men for ignoring the voice of a black woman. Nevermind that it was painfully clear they were writing their responses at the same time. Message after message she would blast both the high profile figure and anyone who so much as implied it could have been a misunderstanding. In the end everyone just bent over backwards apologizing to her and the thread was abandoned along with the whole brain papers group which was relatively active up to that point. She has effectively robbed thousands of colleagues of insights into their seniors thought process just because she didn't immediately get attention.

The thread is still up there so any googler can see it for themselves and verify I am telling the truth.

It's appalling "everyone just bent over backwards apologizing to her ". It shows how detrimental the extreme "progressiveness" can be.

The top voted reply is implied to be from a Googler, and it claims this is inaccurate: https://old.reddit.com/r/MachineLearning/comments/k77sxz/d_t...

From that piece:

> I do believe she actually thinks she is making the world a better place but in reality any interaction with her has been incredibly stressful having to carefully weigh every move made in her presence.

What if what some vocal minority advocates say is correct, and they have to carefully weigh every move they make, throughout their whole lives? If living that carefully is incredibly stressful, what percentage chance of it being the lived experience of minorities in tech justifies someone fighting aggressively to make things easier for minorities?

I agree this is a difficult spot to be put in: be quiet (or nice) and nothing changes, or be loud and aggressive so that people are forced to take notice. I don't wish that kind of double-bind on anybody.

In this position, some people seem to find a third alternative: be fierce and yet unflinchingly kind. Make a lot of noise, but be charitable with your opponents so that their good side has the opportunity to come out. People change on their own time. We can't force it.

Truth be told, it takes considerable wisdom to pull this off, and it's unfair to expect that of most people. But the alternative is to become increasingly bitter and caustic. That ends up making you feel increasingly self-righteous, but also increasingly isolated (which then feeds the self-righteousness, in a vicious cycle). It doesn't actually help the cause.

She doesn't deserve contempt for falling into that trap. Like all of us, she deserves compassion. But it's okay to point out that her behavior is unhelpful. We, too, should not fall into the double bind of keeping quiet about it versus responding aggressively. We can be kind and firm.

I don't know Timnit so I won't argue she is the most toxic or the events were correct/incorrect. However, your reply made me recall this thread from July 2020 (before Timnit and Google parted).

https://twitter.com/timnitGebru/status/1278565265135906816?t...

I just cannot imagine calling someone in your company out like that on a public platform, especially for the crime of not speaking. Both are seniors and that behavior seemed anything but, to me anyway.

I do wonder if mid 2020 will go down in history as peak cancel. I definitely have a sense that it is becoming less acceptable to publicly air one's grievances in the name of social justice. "Silence is violence" is another phrase that was very popular at that time. From my limited perspective, it seems like that same slogan today would be much less acceptable and get much less play.

Of course, two years is far too small a lag to do any real historical analysis of the situation. But it seems like the winds are blowing in a different direction now. I wonder where they will take us. Surely we live in exciting and interesting times.

Calling both "seniors" is a bit of a misnomer, like it's technically true but at the time Jeff was, if memory serves, Timnit's manager's manager's manager (maybe one more?), the level of seniority is not the same.

And the callout was explicitly for (internally) claiming to want to be supportive, but failing to do so externally. It doesn't seem unreasonable in context.

Saying a paraphrase of "I didn't quit, I gave my employer an ultimatum that they then rejected" [0] is a very toxic way to interact with people.

1) Ultimatums in general are not a healthy way to interact with others because they intentionally try to skew power dynamics towards the giver (Do what I say or else...)

2) To then act (upon failure of the ultimatum) as though her employer acted inappropriately implies that she really didn't even give an ultimatum, so much as made a demand that she couched as an ultimatum.

You can agree or disagree about whether she was entitled to act this way (academia is a unique field where toxic behavior like this is often normalized), but I think most people agree that if given the option, they'd prefer to not work with someone who approaches conflict in this way.

[0]https://twitter.com/timnitGebru/status/1334343577044979712?r...

If you cannot give an ultimatum then you can always be pushed around.

> If you cannot give an ultimatum then you can always be pushed around.

No finer example of false dichotomy than this.

Hers is a textbook example of how not to negotiate. By textbook, I mean literally every textbook on negotiations that I've read.

Ultimatums are a tool of last resort, and they all caution their high failure rate. It also is typically a signal of a weak person.

>literally every textbook on negotiations

Aside: can you recommend the best ones? Getting to Yes is all I've read, but it seems a little 'fluffy'. I want something more rigorous, perhaps with game theory and models.

> I want something more rigorous, perhaps with game theory and models.

I haven't read any of those. The thing is, those are great for longer term negotiations, but not for short term day to day ones - you don't have the luxury of evaluating things from a game theory/model perspective. Even Getting To Yes is a bit poor in that regard.

So which ones/resources do you like for day-to-day negotiations?

Unless you stop making comments like that, I will never upvote a comment you write ever again.

^that is certainly more 'toxic' than:

I disagree with your point. I think disagreements and negotiation with an employer are fine (to your point, one can even use the threat of quitting as part of said negotiation), but ultimatums just serve as an opportunity for the giver to imply that they have more power than the receiver, which is 'toxic' behavior

Google is free to publish whatever information they'd like to vindicate themselves. The fact that they don't is information.

Google is also free to sue for libel/defamation, but for some reason choose not to. This is also information.

You can say "well, we don't have the full picture, so we can't draw conclusions", but this will lead to never draw conclusions about the behaviour of anything, as you'll never have a full picture.

And it's strange no one in media questioned Gebru's paper. For instance, her calculation of carbon footprint of training BERT is ridiculous, as she assumes that companies will train BERT 24x7. I think Gebru is a disgrace to the community because she always, I mean literally always, attacks her critics by motives. You think bias is a data problem? You're a bigot (See her dispute with LeCun). You disagree with my assessment on an ML model? You are white male oppressor (her attacking a Google's SVP).

Gebru is not a researcher. She is a modern-age Trofim Lysenko, who politicizes everything and weaponizes political correctness.

That's what kills me. A person in a privileged position inside a company with all the easily-accessed data available to make concrete conclusions about the actual energy consumption of these systems chooses to not learn from that, but instead to publicize wildly inaccurate guesses with no basis in reality. And this person is an "ethicist" somehow.

> And this person is an "ethicist" somehow.

In most cases, the role of ethicists in any field is to justify monstrous behavior by practitioners to outsiders.

In this case, "ethicists" have taken it upon themselves to determine monstrous.

> weaponizes political correctness

I'm sorry to be nitpick-y. I understand what you are saying, but to be clear: Political correctness is a weapon.

The idea is that there is a correct answer (fact), and a separate, unrelated politically advantageous answer (group approval). Following the tactic of political correctness, to any degree, is using the concept for personal gain. There is no way to be political correct without doing so in a harmful manner. Approval of this kind can only be gained by attacking a target.

"So-and-so is using explosives to destroy things." It is the design of the thing.

Political correctness means adherence to a political ideology. Whether something is politically correct or politically incorrect is relative to the political ideology in question. Subjective value statements like "X deserves more funding than Y", which don't have objective answers, may be considered politically correct or incorrect depending on the ideological context being considered.

I'm not sure PC itself a weapon, but I get your point. PC is so ingrained in American culture that we implicitly accept that certain degree of PC is needed.

Fair enough. My perspective is that PC is a destructive act to culture, whether it be American, European, Spanish, etc. I can hardly expect everyone to agree with me on that point.

I suppose my concern with that perspective would be something close to, "Can you extrapolate the difference between common courtesy and political correctness?" I don't know if that sounds insulting somehow - it's not meant to be, but sometimes when I ask those sorts of questions, it badly received.

Anyway, to my mind, there is a difference between a need for politeness & civil discourse, and a perspective which intends to enforce those ideals through destructive language and cultural upheaval.

One is self-imposed, the other an imposition.

Are we talking about the LeCun fight where they were discussing Pulse (the white Obama super-resolution paper)? Didn't Pulse update their paper and show that it was in fact a dataset bias? A lot of these datasets have bias and it is frequently hard to train an unbiased model on a biased dataset. It also isn't the goal for many, though of course there is work on this. Bias is everywhere in a ML system, from the dataset to the model itself. We should be concerned with all these steps but that fight just felt like everyone didn't care about fixing the issue. A lot didn't even consider that the image was decently faithful but that it was our human bias of knowing Obama is black.

The "white Obama" pictures didn't even seem biased. They took a brightly lit photo of Obama and the output face has the same skin tone.

Compare Obama's skin tone in this photo: https://video-images.vice.com/test-uploads/articles/5ef216e9...:*

With other photos. E.g: https://www.obama.org/wp-content/uploads/ar_obama_sq.jpg

Bias is an overloaded word. To be clear, there is bias (there's always bias). Every model always has bias, we just want to understand that to understand the limitations of it. But yes, I agree that the reconstruction is decently faithful. I'm sure an L2 loss isn't that high when comparing the images.

Part of what I was trying to say is that there is a clear human bias when evaluating the reconstruction. The problem here is our prior. We humans already know that the compressed image is Obama because he is a well known face. So we're judging based on that prior. So there is some irony when talking about model bias when you don't recognize the prior bias in evaluation. We should analyze the bias of the model (I mean there's problems with the reconstruction other than the race issue, like the teeth), but we have to also be careful when analyzing because we also have a bias (which we may or may not want to impart upon the model).

The thing that gets me about these bias people in ML is that it is easy to recognize and identify bias. It is much harder to solve them. There seems to be two camps. Those that just want to identify and those that work on resolving them. The former typically say they work in bias and the latter typically says they work on long tail data (or sometimes few/low shot learning, or even metalearning). Yeah, we should recognize biases and discuss them, but just yelling at people doesn't help develop the necessary mathematical frameworks to resolve the issues.

You can't can't use a use a dropper and compare color hex/HSL values on different pictures with different lighting and say "Yup - that's the same", otherwise SFX wouldn't need color grading[1]. The second picture seems to be from a very brightly-lit environment - to the extent of washing out his blue shirt (collar), but the first picture gives no such cues, and appears less well-lit (virtually).

AI models have no implicit knowledge of environmental lighting, and if the training data is exceedingly made of brightly lit images, may end up generating images that belong in well-lit environments (fairer-looking skin tones), but in low-light contexts. When looking at these generated images, the human eye does what it does best: determine relevant gamma-correction using all available cues.

[1]. Assets rendered in the same color under all scene lighting conditions.

I saw an actual Tweet by Gebru that said some loss functions are more racist than others. (I am not making this up.)

Yes, she literally said that.

I read some of her other work, and found little to no substance. She is mediocre at best.

I would personally say that she is not a real AI researcher. (Her prominence, as lent by her CV, is a result of a gamed system).

> I saw an actual Tweet by Gebru that said some loss functions are more racist than others. (I am not making this up.)

Do you have this source? I'm curious what she said. Is this L1 vs L2 but limiting to race?

> it was our human bias of knowing Obama is black

As a side note - Obama is multiracial (white mother). As a multiracial person who makes no effort to fit into American racial categories and whose general attitude toward them is "Your social construction of race is fucked up, man", it astounds me that people would expect an AI to see what's not actually there. Biologically, Barack Obama is half-black and half-white, and probably a whole lot more complicated than that because "black" and "white" are themselves leaky abstractions that we've imposed on the immense variability of human skin tones. Why wouldn't you expect him to autocomplete to a white man 50% of the time and a black man 50% of the time?

Well the goal is that the reconstruction would be "him". Obviously the reconstruction isn't Obama. Limiting the discussion to race isn't a productive means to resolve the errors. There are more than race and they are likely coupled.

>Obviously the reconstruction isn't Obama.

okay, so the model is not a perfect oracle? What's your point, then, if it's not about race?

actually, if you ahve a blurred face of obama, there are many reconstructions that are consistent with that data. His skin is pale, his features are selected from several genetic histories... the whole complaint about that example was overblown.

The quality of her work was also not the best. Just read the AI bias analysis she did. It would not pass scrutiny with an undergrad audience, and yet she does an exaggerated SIGH and acts like she is beyond reproach because she has CREDENTIALS. Nobody cares about credentials. Your study has to stand on its own.

The Times article has some eyebrow-raising details including a written statement from a lead author of the original paper who describes years of harassment and undercutting.

> Ms. Goldie said that Dr. Chatterjee had asked to manage their project in 2019 and that they had declined. When he later criticized it, she said, he could not substantiate his complaints and ignored the evidence they presented in response.

> “Sat Chatterjee has waged a campaign of misinformation against me and Azalia for over two years now,” Ms. Goldie said in a written statement.

> She said the work had been peer-reviewed by Nature, one of the most prestigious scientific publications. And she added that Google had used their methods to build new chips and that these chips were currently used in Google’s computer data centers.

Yet another blot on the Rorschach test, to be sure, but perhaps useful to highlight to consider the total possibility space.

https://www.nytimes.com/2022/05/02/technology/google-fires-a...

Whoever is in the right, wrong, or somewhere in the middle, it's hard not to come away from the NYT article with any impression other than there's been serious internal squabbling and nastiness going on for a good couple of years.

The reasonable supposition is someone higher up just got sick of the whole thing and/or Chatterjee stepped over some line.

It's odd. 95% of the time if a company fired someone (except maybe at the very highest levels) and was asked for comment, they'd say the person is no longer employed by us (or we don't comment on personnel matters) and leave it at that.

And Google Legal profits the most from turning these cases into “personal Rorschach tests.”

In the case of Glasson’s pregnancy discrimination case, she had to wait two years and go through Google-requested health screenings just to reach a settlement. https://www.theguardian.com/technology/2021/apr/09/she-sued-...

Google Legal wants these cases to reach settlement or some other point of deterrence because they don’t want the real evidence and merits to enter public discussion.

Google not just saves reputation value but also counter-party price discovery. Whenever Corp Legal fires an employee like this, they always ask “will this case blow the budget on severance + litigation this year?” They don’t want anybody else to know what this number is, and it could effectively leak if a case doesn’t get settled.

While the linked article may not give many details on the merit of this case, what also is not reported is the realities of large well-funded corporate legal teams.

Agree strongly with you on the Gebru firing. I followed it closely. The Press was pretty dishonest in the way they portrayed it.

https://twitter.com/jacobsteinhardt/status/15215993379432898...

Google AI ethicists and other researchers would be wise to recognize the delta between why Google ostensibly has them, and why Google actually has them. If they aren't tailoring their research to whatever Google executives want to hear, then they aren't doing what they're actually being paid for. And if they want to be honest researchers instead of corporate whores, they picked the wrong company to work for.

Would an honest climatologist last long working for Exxon? Would an honest cancer researcher last long working for Phillip Morris?

Maybe they thought Google was a mostly okay company with good potential and were under the impression that what they were hired for was to help them improve? There was after all a time when many of us really believed Google was a good kind of company and that it generally adhered to the "don't be evil" ethos.

I can see myself falling into a similar trap of wishful thinking, especially if corporate communication reinforces it (until the cold shower wakeup happens).

> Maybe they thought

Pretty sure the thoughts that existed before they viewed their offer letter were different from the thoughts they had afterwards. And there's nothing wrong with it unless you for some reason want to ignore the biggest bias in the room.

What did the offer letter contain?

Big Tobacco, Big Oil, Big Data.

I have no background in ethics research so this is a layman's view but what strikes me as bizarre is that ethics researchers are trying to decide what is ethical. 2 reasonable people can look at the same set of ethical issues, weigh the potential pros and cons and come to different decisions about what course of action is the ethical one. It does not seem to be a field where an ultimate truth exists. I would think an ethics researcher should be examining AI technology to determine the ethical impacts a technology, implementation or use may generate both positive and negative, so that a decision maker can properly weigh them. Once you try to determine what you consider the ethical decision to be and insist that is the only acceptable decision for another to make, you are no longer a researcher but an advocate. Having a belief of what is ethical and advocating for it is fine but it shouldn't be framed as research.

I'm not going to comment on the firing. What I can say, is that all research papers should be questioned heavily. Generally, the review process is very incestuous and creates perverse incentives to not threaten publications. It wouldn't surprise me if firings are related to questioning research.

The article is really light on details. Fired with cause could mean anything.

Business is sometimes a difficult place to do pure research, just as a University is a very poor home for an operating business (outside of education). If you are a business who thinks there is some value in pure research then you could do worse than follow the model from the 50s and 60s. Separate research campus or unit. Separate management. Separate culture and working practices.

What really kills me is that Google claimed previous papers weren't up to their standard, but then ended up citing them in later papers that they've had press events around. It's really unfortunate that so much of our science is being done at large companies rather than universities, since the companies have a bias towards their bottom line which keeps them from publishing things that might damage that.

Which papers are you referring to?

https://www.techrxiv.org/articles/preprint/The_Carbon_Footpr... cites Bender, E.M., et al., 2021, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. ACM Conference on Fairness, Accountability, and Transparency.

In short, Jeff Dean (and others at Google) cited the finally published (not by Google) version of the Parrots paper. The citation [20] is in the context of the paper:

"The opening quote in Section 1 is based on a 2019 study from the University of Massachusetts (UMass) that estimated the environmental impact of training [2]. More than 1000 papers cite this paper as the source for the impact on carbon emissions of ML models, e.g., [1,6,7,17,18,19,20]."

in which the authors of this paper point out that the Parrots paper used what is likely a large overestimate of the carbon emissions from training. This is consistent with my understanding (that the power/emissions part of the Parrots paper isn't well backed by evidence).

Ahh thank you for the context. I was familiar with the Timnit Gebru incident, but I haven't followed up on it since it happened.

But...but...there's nothing private investors and the free market can't do!

Seriously, who thought that science for profit would net the best results for the collective benefit?

You take science but replace truth with profit as the ultimate objective. I'm sure that's going to work well.

The day we stop seeking profit in endeavors that can only be pursued properly at a complete economic loss, we're going to solve so many problems at once.

> Seriously, who thought that science for profit would net the best results for the collective benefit?

Nobody, and I don’t think “best results” or “collective benefit” were the goal when science started getting more commercialized.

The idea that good inventions would benefit all rather than patent holders must drive a true capitalist crazy.

Yet even Adam Smith bases his entire theory on the supposed collective benefit of the invisible hand. Without collective benefit, there's nothing positive left to it.

Universities have their own biases as well --whether it's to pad one's CV, the PI's CV, or whatever, but they are also not free from interference against 'pure' research.

There is a lot of "garbage" research that happens at universities. Yes, we do expect quite a bit of research to be dead-ends --that's the nature of research, but there is a lot of research that is knowingly fraudulent or uses very suspect data and or methodology.

Out of curiosity, are there any institutions that meets a “pure” research standard? NIST?

Or is that standard unattainable because undue influence exists everywhere?

Influence exists everywhere because without influence the studies wouldn't exist in the first place.

And people need careers, successful publications and grants make that possible.

That isn’t to say everything is trash— lots of good stuff is still done with these requirements.

Unless you have a wealthy aristocracy and some of them are bored so they create a lot of the major science of past couple of centuries.

Get a private wealthy benefactor that likes your work, then you can avoid undue influence.

..until the wealthy benefactor decides to exert it anyway

Now the influence just comes from the wealthy benefactor and his interests

Works best when the benefactor is your dead parents.

That's in principle what academic tenure is for, it guarantees independence and research freedom. And while you may talk about how there's always this or that political topic that anyone will shy away from, at least in the hard sciences I think academic freedom is largely intact.

In a company you can always be fired and the singular goal is the bottom line. Much more so since modern companies seem to not operate with the mentality of the old corporate research labs like Bell, but keep a much tighter lid on things, always pushing for products.

While true, are these biases impactful in the same ways?

Bias will always exist. The nature of that bias and the degree of impact will vary.

What papers are you talking about specifically? Were they reworked at all after being told they weren't up to standard? Does Google have a standard where only papers they would publish can be cited?

Imagine if Google had a standard that all papers they publish must be in Comic Sans font. Researcher refuses, publishes paper independently, and then someone at Google publishes a paper in Comic Sans referring to that paper. Is that wrong? After all, Google doesn't have a standard that all cited papers must be published in Comic Sans as well.

The controversy here is specifically around Stochastic Parrots, by among others Timnit Gebru and "Schmargaret Smitchell". Timnit had recently been fired by Google for a controversy that started when leadership prevented her and Margaret from being coauthors on the paper (which is critical of Google in some ways) and Mitchell was fired a few months later.

Google[0] claims the paper didn't meet the bar for publication and ignored too much relevant research. The authors obviously dispute these claims (I generally side with the authors that the "ignored research" line is a stretch).

The then amusing thing is that a number of Google authored or coauthored papers still cite it, for various reasons[1][2] (but there's another like half dozen or so Google papers at least in the ~500 citations). So the entire controversy is that the paper wasn't up to academic standards, except that empirically it clearly was.

> Were they reworked at all after being told they weren't up to standard?

There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively.

[0]: https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ... [1]: https://arxiv.org/pdf/2107.06499.pdf [2]: https://arxiv.org/pdf/2109.01652.pdf

If the argument is Timnit was fired for her Stochastic Parrots paper, why weren't the Google authors that referred to her papers fired as well?

Instead, maybe Timnit was fired/resignation accepted not because of her paper, but because 1) she was in management, and 2) she explicitly told her reports to stop the work they're doing, for "reasons", to the point where Jeff Dean had to come in and ask the employees to continue working on DEI programs: https://www.platformer.news/p/the-withering-email-that-got-a...

I simply repeated Jeff Dean's own words that he claims the paper "didn’t meet our bar", yet continued to be cited (even by Dean himself, though arguably he's citing it to disagree...but then that's the way academia should work!).

I didn't claim to know why exactly she was fired. I'm just claiming that Jeff's objection to her paper seems dubious.

And keep in mind her coauthor (Mitchell) was eventually fired. But yes, the question you ask is a good one, and I think the answer is revealing (the paper did in fact meet the bar for publication, and it was killed for political reasons, making Gebru's reactions quite reasonable!)

> she explicitly told her reports to stop the work they're doing,

No she didn't. She told them to refocus to be more effective. That's a completely reasonable! Quoting her email: "So if you would like to change things, I suggest focusing on leadership accountability and thinking through what types of pressures can also be applied from the outside."

That doesn't sound like "stop working on DEI" to me.

Does "What I want to say is stop writing your documents because it doesn’t make a difference" sound like stop working on DEI to you? With the context that the documents are their work products about DEI...

With the full context of the letter, no not at all, unless you think that Documents are the ultimate work product of DEI and not like improvements in diversity, equity, and inclusion in the company.

It's very clearly a call to stop focusing on ineffective things and to focus on more effective methods.

> There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively.

Really? Because people claiming to be the reviewers at the time said that the version they reviewed was a lot worse than the final version published much later. (And that version wasn't too great either.)

That's because the paper received additional external peer review as part of the normal academic publishing process at FAccT.

The internal special topics review process that wasn't formalized at the time but was pseudo-formalized later in response to this issue also supposedly allowed revision, but in practice this authors weren't given the opportunity to respond to the feedback, they were only give the option to remove their names from the paper.

So both your statement and mine can be true, as their were multiple reviewers in multiple review processes. To further disambiguate, the FAccT reviewers have noted that the paper improved during the academic peer review process. This is likely true. But that's wholly irrelevant to the line of discussion I was having, because that criticism wasn't "internal", and we can be sure it wasn't relevant because we know the internal reviewers have never identified themselves or made comment on the internal review process.

I recognize that this is complicated, and it doesn't help that afaict, Google has tried to blur the lines between their process and the normal academic peer review process.

Well OK then, but your point was that "the original version must have been acceptable because Googlers later cited the final version, gotcha!", and the more review and improvement it undergoes by whomever, the less sense your point makes.

You're correct that that claim doesn't make sense, but then it's not one I made.

Can't you cite things that you disagree with? Or cite background material that you feel is subpar?

That would be the norm especially if your research paper is focused on new technology. For example you would write Herbet et co, in Journal Blah found results of X with method Y. However method Y has flaws of A, B, and C...

Even if a pervious study was utterly stupid in your opinion, if it's been widely publicized you still need to talk about it or else risk being accused of missing the basics.

This was my thought as well. Merely citing the material isn’t necessarily a problem on its own.

The nature of the citation and how that impacts the paper doing the citing would need to be examined to form a meaningful opinion about Google’s behavior here.

You are right but the implication was that the citations were not rebuttals.

Then the answer is that even papers that don't meet the "high" standard that every organization and journal likes to think they hold themselves to when publishing, still contain relevant data and explanations that are citable.

Although let's be clear... this is bureaucratic issue. Some journals only publish English language papers. No matter how impactful your paper is, they'll never publish it if you wrote it in Tamil. Google only publishes papers that doesn't directly insult them, and spread fear about their own proprietary technology while providing no clear solutions. The paper may be brilliant but they definitely won't put their money and name behind it.

Since they (Google) are ideologically opposed to the paper's findings, I can't imagine they are citing it to talk about how brilliant it is though...

Do you have an example of Google citing a "below standard" paper? Curious to see.

I mean, for all we know, those papers could have been cited as "look here's how not to do it or here's how we're different/better". There are positive and negative citations in papers.

From an author's perspective, citing a paper is not an endorsement.

For example, a paper's intro might have an overview of how others have attempted to solve a problem before. Citations there will comprise a very concise summary of techniques from others, often pointing out limitations.

From the perspective of a PR team or agency, they aren't in a position to independently evaluate papers, let alone disavow a paper that carries the name of the company.

Papers may not meet the standards for publication but still be useful. For example, I have a paper with 50 citations that is having difficulties getting published (either "not novel enough" or "not motivating"). There's also a randomness to review, as NeuralIPS has continually demonstrated.

I should also mention that you can also pad your citations this way and build your h-index. It is metric hacking, but if it isn't too egregious this is normal. Classic example of Goodhart's law.

wth? a "for cause" termination is a pretty big deal (in some larger organizations practically an automatic application killer), and to say it in a press statement is furthering its damaging impact. why do they think they need to do that?

even if we take for granted that there's probably a lot more to this story than what's reported in the article, this leaves a bitter aftertaste. there are so many ways this could have been handled more gracefully. and the optics of doing that in the context of a critical study are poor, to say the least.

Archived copy without GDPR nag screen:

https://web.archive.org/web/20220502154044/https://www.engad...

If you seek truth, search on moma or ask for “Experiments for Nature rebuttal” document. Read the document and come back for comments :)

Why does Google even fund this AI safety research internally? These conflicts are guaranteed to happen.

Checks a box that outside regulators can look at. Otherwise outside regulators would likely try to investigate and regulate. Also Google's own research can be used to counteract AI ethics claims that are existential to Google. These are likely going to be Big Data type claims. Big ominous future threats like the GDPR for example.

We know the oil and gas industry produce copious amounts of CO2 causes global warming -- they knew because they funded the research. But publicly their position was "uncertainty".

What a way to bias the network.

"Google fired machine learning scientist Satrajit Chatterjee in March, soon after it refused to publish a paper Chatterjee and others wrote challenging earlier findings that computers could design some chip components more effectively than humans. The scientist was reportedly allowed to collaborate on a paper disputing those claims after he and fellow authors expressed reservations, but was dismissed after a resolution committee rejected the paper and the researchers hoped to bring the issue to CEO Sundar Pichai and Alphabet's board of directors."

Google refused to allow a paper that was critical of a previous paper to be published? That would be scientific misconduct at the organization level, no? The kind of thing that got the Trump administration in trouble, right?

It is a strange hill to decide to die on. Even if you are the expert in the area, AI is weird and changing the world.

Most Nature papers have something wrong with them. I don't see any reason that reinforcement learning cannot learn the best layouts or what not. I don't know the space though enough to say more than that.

The NYTimes article alleges that he waged a misinformation campaign against two researchers. So there's that in the mix as well.

They can fire him for whatever reason they like, but preventing the publication of a paper should not be a corporate policy decision. The only question they should ask is if it reveals trade secrets, which I assume it wouldn't because it was a response to another paper from Google. The whole scientific process thing is the part intended to decide if it's meaningful or not.

That AI is weird and changing the world is why the science shouldn't be held up on the question of whether it embarrasses the company.

How can I get on the AI shyster gravy train?

Tangentially: who decided to make the links all purple, horrible ux.

In the corporate world, just like government funded research, science is only science if it aligns with the desires of its funding source unfortunately.

I never experienced anything like this during the course of my government funded research. Nor did I hear of anyone else experiencing anything like this. Of course, your topic area has to align with government funders’ interests… but in my experience the conclusions do not.

But if your conclusions do not align with the funders desires, would your research see the light of day?

If you have that much money that you can fund lots of studies, the only ones that will proceed are the ones in support of whatever-it-is.

> But if your conclusions do not align with the funders desires, would your research see the light of day?

If you are in academic science doing publicly funded academic research, as most academics do, then yes, your funding agency does want you to get the results published, period. The conspiracy theories you've been led to believe are just that, conspiracy theories, not the objective realities of those doing the work.

It's not a matter of publishing, it's a matter of funding the research in the first place. What academic field are you in where this isn't a problem? Even something as seemingly inconspicuous as physics research is highly political.

Researchers know the ideas that one better doesn't question if one cares about job security. Or put in a more positive way: You better look into -current list of hype topics- if you want to continue getting funded, being cited and staying relevant. The problem with "academic science" as you call it is that it's mostly academic and not enough science.

Funding is always going to be preferentially available for the things that people are interested in. That's true in all aspects of life, not just in science. If you want to challenge the Standard Model in Physics, of course you're going to have to work harder to get funding. That's because the betting money is of the opinion it's probably at least mostly correct. That rational allocation of funds bias doesn't prevent you from doing research or from getting it published, it just says that yes, if you want to challenge the prevailing wisdom you're going to have to work pretty hard to pull it off and convince people the standard model of the field is wrong.

If your topic area aligns with your government funders interests but your conclusions do not, it usually means you were working on a scientific dead end anyway

> But if your conclusions do not align with the funders desires, would your research see the light of day?

In academia: Yes. The number of grants that require pre-publishing approval from funding agency is extremely tiny. I never encountered one while in academia.

I take it you never had a grant director run afoul of one of the consulting firms on I Street who didn't get the follow-on evaluation contract? 25 years and I'm still a bit peeved about how politics and who gets what makes government research such a pain in the butt. Every agency has its prejudices also which leads to some shenanigans.

Sorta off topic tip: if you are writing government grants and are in a position to become a reader, then do so. It gives you an idea how they thinks, tells you what words / wording will get your grant down-voted, and it gives you some good contacts. Had relatives do that, but went a different career direction.

What?

Grants are often government originated in the US and politics and people can affect the results and funding.

You may want to spend less time reading anti-intellectual media and more time talking to people who actually do academic scientific research. There are many challenges in academic science, but it does a remarkably good job of not falling prey to the strawman risks you believe in. Corporate research is of course subject to the economic interests of the corporation. Academic research is not held back from publishing in the same way.

Does it? I picked the first professor from MIT's CSAIL group (Hal Abelson), found their latest published paper (https://www.erudit.org/en/journals/irrodl/2022-v23-n1-irrodl...) and found that they are funded by a Chinese governmental organization. Tell me how the research isn't being influenced by the needs and desires of the funder?

MIT CSAIL is not hard up for funding, nor is Hal Abelson. They can choose who they want to take money from. Funding sources that restrain a researcher's ability to publish damage that researcher's ability to advance their own career.

Many people here on HN have, for example, an irrational hatred and distrust of Microsoft. Microsoft Research, however is one of the biggest sources of computer science research funding in the world. If MSR funds you, they expect your work to result in publications and the primary factor in whether you get additional funding is whether your papers impact your field. Those characteristics make MSR funding highly desirable, precisely because it comes with no strings attached.

“Many people here on HN have, for example, an irrational hatred and distrust of Microsoft”.

Perhaps. Many people here on HN also have an entirely rational hatred and distrust of Microsoft. Some of those people are also capable of appreciating their support for research.

> Many people here on HN also have an entirely rational hatred and distrust of Microsoft. Some of those people are also capable of appreciating their support for research.

I am standing right here, you dont have to point.

Taiwan governmental organization (note, the coauthor who received the funding works in Taiwan). Details matter- Taiwan and China are two completely different entities.

I agree Taiwan and China are different entities, looking at the acknowledgement of the paper: "This study was supported in part by the Ministry of Science and Technology under contract numbers MOST 108-2511-H-003 -056 -MY3 and by the Hong Kong Jockey Club Charities Trust". I read Hong Kong and assumed it was China.

This is a laughably bad take, sorry. For example, the University of Chicago and its connected education institutions get a large portion of their funding by falling to the interests of the Chicago's mayor's office and the police department, which largely stems from rhetoric. They wouldn't be able to fund their projects if they didn't do that. Academic researchers can lose their funding if they, as individuals, publicly criticize certain organizations.

In other words, "validate our political rhetoric with a smile, or lose your funding." It skews academic research, and these incentives exist practically everywhere.

The University of Chicago is a private institution with a research budget of many billions of dollars per year. Given the extreme difficulty of believing the government of the city of Chicago provides even a tiny fraction of those billions of dollars in research funding, I'll have to ask you for citations or data to support those aspersions you're casting on the university and it's researchers.

We live in a time when there are strong anti-intellectual threads and forces among some parts of the media. It's easy to cast aspersions against researchers. The fact that you've heard someone in the media say these things doesn't make the aspersions true.

  The fact that you've heard someone in the media say these things doesn't make the aspersions true.

Good one. I'm a journalist who's done extensive research on this exact subject in Chicago and have spent hundreds of hours spent looking through leaked documents, FOIA request documents, network graphs, etc etc. I'm not going to write a long post about the subject for you, but I can point you in the right direction if you're actually curious about diving into this. I hope you are, because you seem to be very resistant to even attempting to believe.

A lot of it is hush-hush, mired in layers of obfuscation, but one of the better places to start is by looking into the funding of the University of Chicago's "Crime Lab" in connection with the Chicago Police Department's SDSC program.

https://news.uchicago.edu/story/10-million-grant-will-suppor...

Leaked documents between the Chicago Mayor's office and executive staff at University of Chicago is a great resource as well.

https://www.documentcloud.org/documents/20639065-re_-quiet-t...

https://www.documentcloud.org/documents/20651206-re_-fundrai...

Do a search for "uchicago.org" here: https://www.documentcloud.org/app?q=%2Bproject%3Aemails-2028...

And have a look at how often Chicago's worked directly with the Uchicago over the past 15 years: https://observablehq.com/d/22a60bb00bade634

The funding organization that threatened to cut funding of a program after tweets were made can't be shared because they were deleted after they were bullied by their funders.

I'm not seeing the smoking gun here in your documents.

I'm aware of many academics who work with city governments, including at least one researcher not from Chicago who works closely with the city of Chicago. He does so precisely because there are big problems there. The city of Chicago may even contribute to the funding of the work he does (it's helpful for the city to have skin in the game), but they certainly don't provide the lion's share of the funding and if the city were uncooperative he'd simply shift his work to another urban area. He's not going to waste his time and career as some sort of controlled victim of an evil governmental moneybags, particularly when he's the one bringing funding to the projects.

Happy to continue reading docs, but thus far I'm not seeing the conspiracy.

The point I'm hoping to show isn't a "smoking gun" (frankly, I doubt any single document, or set of documents, would meet your criteria), but to show that research is deeply motivated by funding rather than for "pure science". Bad results that don't fit the narrative of the funders or political body's interest either don't get published, or are downplayed to the extent that they're effectively unpublished. Anything remotely positive is exploded through the roof.

I recommend checking out Robert Vargas on twitter. He's a tenured professor at University of Chicago who's written a lot about this subject. https://twitter.com/robvargas21

This post describes a lot of the problems well: https://crimethoughts.com/year/04/24/readi-chicago-and-incon...

Reading your second set of links, it sounds like much more a case of "the media does a bad job of reporting on science" than "the research is corrupt." If you want to improve the quality of science reporting, that's probably better handled by criticizing the reporters than by criticizing the researchers (most researchers already work hard to keep their work out of the popular press, precisely because the popular press does such a bad job of reporting on research).

Among the first tweets in the twitter stream you provided is a University of Chicago researcher actively criticizing the actions of the Chicago PD. That doesn't seem to support your narrative that the U of C research is corrupt and in the pocket of city government. I certainly can believe research is being poorly and inaccurately reported in the press, but again I think if you want to affect change you'll be more effective if you focus your efforts on the reporting press who seem by your own description to be the ones at fault.

You're reading things at a very surface level, friend. Good luck to you and yours.

Reply on news.ycombinator.com