Hacker News Reader

Comment by tedivm | Hacker News Reader

What really kills me is that Google claimed previous papers weren't up to their standard, but then ended up citing them in later papers that they've had press events around. It's really unfortunate that so much of our science is being done at large companies rather than universities, since the companies have a bias towards their bottom line which keeps them from publishing things that might damage that.

View on HN

Universities have their own biases as well --whether it's to pad one's CV, the PI's CV, or whatever, but they are also not free from interference against 'pure' research.

There is a lot of "garbage" research that happens at universities. Yes, we do expect quite a bit of research to be dead-ends --that's the nature of research, but there is a lot of research that is knowingly fraudulent or uses very suspect data and or methodology.

Out of curiosity, are there any institutions that meets a “pure” research standard? NIST?

Or is that standard unattainable because undue influence exists everywhere?

Influence exists everywhere because without influence the studies wouldn't exist in the first place.

And people need careers, successful publications and grants make that possible.

That isn’t to say everything is trash— lots of good stuff is still done with these requirements.

Unless you have a wealthy aristocracy and some of them are bored so they create a lot of the major science of past couple of centuries.

Get a private wealthy benefactor that likes your work, then you can avoid undue influence.

..until the wealthy benefactor decides to exert it anyway

Now the influence just comes from the wealthy benefactor and his interests

Works best when the benefactor is your dead parents.

That's in principle what academic tenure is for, it guarantees independence and research freedom. And while you may talk about how there's always this or that political topic that anyone will shy away from, at least in the hard sciences I think academic freedom is largely intact.

In a company you can always be fired and the singular goal is the bottom line. Much more so since modern companies seem to not operate with the mentality of the old corporate research labs like Bell, but keep a much tighter lid on things, always pushing for products.

While true, are these biases impactful in the same ways?

Bias will always exist. The nature of that bias and the degree of impact will vary.

Can't you cite things that you disagree with? Or cite background material that you feel is subpar?

That would be the norm especially if your research paper is focused on new technology. For example you would write Herbet et co, in Journal Blah found results of X with method Y. However method Y has flaws of A, B, and C...

Even if a pervious study was utterly stupid in your opinion, if it's been widely publicized you still need to talk about it or else risk being accused of missing the basics.

This was my thought as well. Merely citing the material isn’t necessarily a problem on its own.

The nature of the citation and how that impacts the paper doing the citing would need to be examined to form a meaningful opinion about Google’s behavior here.

You are right but the implication was that the citations were not rebuttals.

Then the answer is that even papers that don't meet the "high" standard that every organization and journal likes to think they hold themselves to when publishing, still contain relevant data and explanations that are citable.

Although let's be clear... this is bureaucratic issue. Some journals only publish English language papers. No matter how impactful your paper is, they'll never publish it if you wrote it in Tamil. Google only publishes papers that doesn't directly insult them, and spread fear about their own proprietary technology while providing no clear solutions. The paper may be brilliant but they definitely won't put their money and name behind it.

Since they (Google) are ideologically opposed to the paper's findings, I can't imagine they are citing it to talk about how brilliant it is though...

From an author's perspective, citing a paper is not an endorsement.

For example, a paper's intro might have an overview of how others have attempted to solve a problem before. Citations there will comprise a very concise summary of techniques from others, often pointing out limitations.

From the perspective of a PR team or agency, they aren't in a position to independently evaluate papers, let alone disavow a paper that carries the name of the company.

I mean, for all we know, those papers could have been cited as "look here's how not to do it or here's how we're different/better". There are positive and negative citations in papers.

Do you have an example of Google citing a "below standard" paper? Curious to see.

Papers may not meet the standards for publication but still be useful. For example, I have a paper with 50 citations that is having difficulties getting published (either "not novel enough" or "not motivating"). There's also a randomness to review, as NeuralIPS has continually demonstrated.

I should also mention that you can also pad your citations this way and build your h-index. It is metric hacking, but if it isn't too egregious this is normal. Classic example of Goodhart's law.

What papers are you talking about specifically? Were they reworked at all after being told they weren't up to standard? Does Google have a standard where only papers they would publish can be cited?

Imagine if Google had a standard that all papers they publish must be in Comic Sans font. Researcher refuses, publishes paper independently, and then someone at Google publishes a paper in Comic Sans referring to that paper. Is that wrong? After all, Google doesn't have a standard that all cited papers must be published in Comic Sans as well.

The controversy here is specifically around Stochastic Parrots, by among others Timnit Gebru and "Schmargaret Smitchell". Timnit had recently been fired by Google for a controversy that started when leadership prevented her and Margaret from being coauthors on the paper (which is critical of Google in some ways) and Mitchell was fired a few months later.

Google[0] claims the paper didn't meet the bar for publication and ignored too much relevant research. The authors obviously dispute these claims (I generally side with the authors that the "ignored research" line is a stretch).

The then amusing thing is that a number of Google authored or coauthored papers still cite it, for various reasons[1][2] (but there's another like half dozen or so Google papers at least in the ~500 citations). So the entire controversy is that the paper wasn't up to academic standards, except that empirically it clearly was.

> Were they reworked at all after being told they weren't up to standard?

There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively.

[0]: https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ... [1]: https://arxiv.org/pdf/2107.06499.pdf [2]: https://arxiv.org/pdf/2109.01652.pdf

If the argument is Timnit was fired for her Stochastic Parrots paper, why weren't the Google authors that referred to her papers fired as well?

Instead, maybe Timnit was fired/resignation accepted not because of her paper, but because 1) she was in management, and 2) she explicitly told her reports to stop the work they're doing, for "reasons", to the point where Jeff Dean had to come in and ask the employees to continue working on DEI programs: https://www.platformer.news/p/the-withering-email-that-got-a...

I simply repeated Jeff Dean's own words that he claims the paper "didn’t meet our bar", yet continued to be cited (even by Dean himself, though arguably he's citing it to disagree...but then that's the way academia should work!).

I didn't claim to know why exactly she was fired. I'm just claiming that Jeff's objection to her paper seems dubious.

And keep in mind her coauthor (Mitchell) was eventually fired. But yes, the question you ask is a good one, and I think the answer is revealing (the paper did in fact meet the bar for publication, and it was killed for political reasons, making Gebru's reactions quite reasonable!)

> she explicitly told her reports to stop the work they're doing,

No she didn't. She told them to refocus to be more effective. That's a completely reasonable! Quoting her email: "So if you would like to change things, I suggest focusing on leadership accountability and thinking through what types of pressures can also be applied from the outside."

That doesn't sound like "stop working on DEI" to me.

Does "What I want to say is stop writing your documents because it doesn’t make a difference" sound like stop working on DEI to you? With the context that the documents are their work products about DEI...

With the full context of the letter, no not at all, unless you think that Documents are the ultimate work product of DEI and not like improvements in diversity, equity, and inclusion in the company.

It's very clearly a call to stop focusing on ineffective things and to focus on more effective methods.

> There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively.

Really? Because people claiming to be the reviewers at the time said that the version they reviewed was a lot worse than the final version published much later. (And that version wasn't too great either.)

That's because the paper received additional external peer review as part of the normal academic publishing process at FAccT.

The internal special topics review process that wasn't formalized at the time but was pseudo-formalized later in response to this issue also supposedly allowed revision, but in practice this authors weren't given the opportunity to respond to the feedback, they were only give the option to remove their names from the paper.

So both your statement and mine can be true, as their were multiple reviewers in multiple review processes. To further disambiguate, the FAccT reviewers have noted that the paper improved during the academic peer review process. This is likely true. But that's wholly irrelevant to the line of discussion I was having, because that criticism wasn't "internal", and we can be sure it wasn't relevant because we know the internal reviewers have never identified themselves or made comment on the internal review process.

I recognize that this is complicated, and it doesn't help that afaict, Google has tried to blur the lines between their process and the normal academic peer review process.

Well OK then, but your point was that "the original version must have been acceptable because Googlers later cited the final version, gotcha!", and the more review and improvement it undergoes by whomever, the less sense your point makes.

You're correct that that claim doesn't make sense, but then it's not one I made.

Which papers are you referring to?

https://www.techrxiv.org/articles/preprint/The_Carbon_Footpr... cites Bender, E.M., et al., 2021, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. ACM Conference on Fairness, Accountability, and Transparency.

In short, Jeff Dean (and others at Google) cited the finally published (not by Google) version of the Parrots paper. The citation [20] is in the context of the paper:

"The opening quote in Section 1 is based on a 2019 study from the University of Massachusetts (UMass) that estimated the environmental impact of training [2]. More than 1000 papers cite this paper as the source for the impact on carbon emissions of ML models, e.g., [1,6,7,17,18,19,20]."

in which the authors of this paper point out that the Parrots paper used what is likely a large overestimate of the carbon emissions from training. This is consistent with my understanding (that the power/emissions part of the Parrots paper isn't well backed by evidence).

Ahh thank you for the context. I was familiar with the Timnit Gebru incident, but I haven't followed up on it since it happened.

But...but...there's nothing private investors and the free market can't do!

Seriously, who thought that science for profit would net the best results for the collective benefit?

You take science but replace truth with profit as the ultimate objective. I'm sure that's going to work well.

The day we stop seeking profit in endeavors that can only be pursued properly at a complete economic loss, we're going to solve so many problems at once.

> Seriously, who thought that science for profit would net the best results for the collective benefit?

Nobody, and I don’t think “best results” or “collective benefit” were the goal when science started getting more commercialized.

The idea that good inventions would benefit all rather than patent holders must drive a true capitalist crazy.

Yet even Adam Smith bases his entire theory on the supposed collective benefit of the invisible hand. Without collective benefit, there's nothing positive left to it.

Reply on news.ycombinator.com