Google told its scientists to ‘strike a positive tone’ in AI research | Hacker News Reader

Google told its scientists to ‘strike a positive tone’ in AI research

reuters.com

303 pointspseudolus271 comments

I deal with this on a regular basis in the research world, there is what is a really toxic and dangerous bias towards constructing the positive, i.e., making things seem, appear, and even fabricate outcomes that will be affirming of desired outcomes or other group biases.

It is not research, let alone science when you are striking a positive tone, it’s fraud.

This is a really important point. Executive-land is all about image and framing, so I can totally see how execs think it's not a big deal to "just make the case" or frame things less negatively.

But while that's ok in a slide deck, it's _not_ ok in a research paper. Researchers aren't (shouldn't be) accountable to business outcomes, but rather the truth of what they argue.

It's tough to see this play out real time.

Grant-land is very very often about image and framing. How often are grant administrators interested in pure research and let the chips fall where they may vs getting results that yield happy articles, then more grants?

Agreed, the bias towards culling negative results is not unique to Google, although the mechanism may be

There is a witch hunt against Google that is politically motivated happening now.

If Google is funding its own research, it should be able to look at whatever it wants. Why do we hold Google to a standard that most other companies are far worse at following?

The appropriate place for independent critical research is outside of the company itself.

Google is actually a pretty well intentioned entity to put out as much self critical research as it does.

Remember, it's all about the "optics".

This isn't relegated to science or management - we're seeing it all across society these days. Nuance is dead. If the message cannot be delivered in a 120 character tweet, and cannot self-gratify 50 million teenage viewers with only a superficial understanding of the topic, then it shouldn't be a public position at all. Of course, I'm slightly exaggerating.

But the consequence is very real - organizations, whether corporate or scientific, whether public or private have had to drop an iron curtain around their honest nuanced internal strategies in order to avoid the attention of the hyperbolic and outrage-fueled social media world.

> Researchers aren't (shouldn't be) accountable to business outcomes, but rather the truth of what they argue.

if a business funds research, they expect the funding to have a positive ROI. Therefore, the business will pressure research towards this, and thus, will inevitably make the research accountable for an outcome.

This is why fundamental research should be publicly funded, and publicly available.

indeed. this just reads like managing PR, as opposed to working with researchers to i don't know, attempt to remove racial bias for instance.

That isn’t just true in the research world, it’s even worse in the corporate world. Something about human nature where naysayers don’t get much acceptance.

From what the article mentions, it looks like the sensitive subjects they worry about are those which can get a company burned at the stake on social media these days. It's not great, but it's un unsurprising effect of the current culture wars.

Or annoy the Chinese government, potentially make their self-driving cars project look less perfect, make it a bit too obvious how much location tracking Google is doing, make their home security integration plans not quite so shiny, and so forth. The actual list is very clearly not just about social media, if anything the majority is topics Google has a financial interest in that don't get that much attention on the big networks.

bit of slippery cliff you've tripped over there. From "strike a positive tone" to "fabricate outcomes" (data?). The latter is clearly fraud, and qualitatively different from the Google manager's suggestion.

Every specialty has taboos and learning what they are and how to avoid them is part of being successful with the job. As a physics graduate student I made an impassioned case to a high energy researcher that the money spent on high energy research was low yield and should be pushed to the biological sciences ... suffice it to say he did not take my points on their merits. Upton Sinclair’s famous “ It is difficult to get a man to understand something, when his salary depends on his not understanding it.” applies in the sciences.

Reminds me of an interview a cosmologist from my university gave to the CBC around the time the higgs boson was confirmed by the CERN. When asked about real world consequences of the discovery, he answered "None."

I mean Heinrich Hertz, who proved Maxwell's electromagnetic wave theory also said "It's of no use whatsoever[...] this is just an experiment that proves Maestro Maxwell was right—we just have these mysterious electromagnetic waves that we cannot see with the naked eye. But they are there." [1]

How wrong he was.

[1] https://en.wikipedia.org/wiki/Heinrich_Hertz

If you had asked Erwin Schrödinger how his wave equation could be commercialized he'd probably wouldn't have had any idea either. 100 years later it's the basis for most of our advanced computer technology. I predict the same thing will happen with the standard model, it's just that we can't see the applications yet because we're not technologically advanced enough.

This seems very different to me than hiring researchers doing work in fields that directly impact your business and telling them their work will not be censored. And then censoring them from a position of power, since you now provide their paycheck.

Keep mind that this is a new, recently instituted process. During the interview process when these researchers were hired, this policy did not exist. Perhaps they were naive to believe that such a policy wouldn't be instituted later but I think that's a different discussion.

The policy existed, management just assumed that they would implicitly understand that they shouldn’t work on research that will knee-cap Google.

Of course, they also hired unhinged woke employees so it’s no surprised they’re reaping the seeds they sowed

The Reuters article says exactly the opposite. If you have documentation that this policy has been in place but ignored, I think that would have a big impact on this unfolding story, it definitely strike me as newsworthy.

The Reuters article clearly refers to this as a new change in policy and procedure.

"Google’s new review procedure asks that researchers consult with legal, policy and public relations teams before pursuing topics such as face and sentiment analysis and categorizations of race, gender or political affiliation, according to internal webpages explaining the policy."

"Studying Google services for biases is among the “sensitive topics” under the company’s new policy, according to an internal webpage. Among dozens of other “sensitive topics” listed were the oil industry, China, Iran, Israel, COVID-19, home security, insurance, location data, religion, self-driving vehicles, telecoms and systems that recommend or personalize web content."

To be clear, there was a review policy prior to the new one, but it's focus was rather different, the primary concern was on not revealing proprietary information. Which is a fairly reasonable restriction, though I would be interested to know what, if any, papers were asked to be withdrawn or retracted under the old policy.

I am a physicist, and I would say that, perhaps, you don't understand why we "waste" money on high energy physics. It's very similar to the reason why rich people buy useless status symbols, when they could instead buy useful things. It's essentially telling everyone, "I am so rich I don't care about money anymore and can afford to waste some of it on this useless thing".

In science, we as a society throw money at something like high energy physics, because we are telling ourselves, "We love knowledge so much, that we will throw it at high energy physics, which has little economic benefit to us" [1].

In other words, you can't prove that you love knowledge for its own sake, if you only invest in fields which have economic benefits. You have to invest in fields with no benefit, to prove to ourselves that we still care. That has deep and lasting philosophical impact on society.

[1] If you read grand funding agencies internal documents, you will learn that the primary economic benefit of fields like high energy physics is training people on hard problems, who will then go and apply those skills and raise the bar in other fields.

Your [1] should add that “additionally, there is a huge funding funnel that select companies can tap to without having to produce any results at all!” So funneling money to friends and special (dark) projects is also on the table, as a simple matter of what is “possible”.

How did you know, as a graduate student no less, that research money spent on high energy research was low yield compared to the biological sciences? I think that's a pretty bold and naive statement to make.

For example, research on quantum-mechanical light-matter interaction was considered not very practically relevant in the 80s and 90s. In the 2000s the results were picked up by the superconducting qubit research community and used to great effect to build a viable approach to quantum computing.

The outcome of research is inherently hard to predict, and telling people that their field of study is "low yield" because it's not the hot topic of the day seems quite naive and arrogant to me.

I’ll grant you the boldness, naïveté and arrogance of my 2006 self - he could be an insufferable a$$. But looking from the vantage point of 14 years on I think the statement is accurate and that researcher understood why. There was simply nothing on the horizon that remotely held as much promise as Alain Aspect’s and Serge Haroche’s quantum optics work.

Just because I was a graduate student does not automatically mean I was wrong.

The difference is that some of the biological research money ends up going to discoveries that dramatically improve the human condition, while quantum computers don't have any path to doing so yet (while conventional computers solve most of the important problems we have today).

Even biology has a pretty low dollar-to-discovery rate, though.

There are many comments that identify the work by corporate sponsored researchers as fraud and not research.

I have the unpopular opinion, and I vehemently disagree.

There is no altruistic, sponsored research, no matter of the sponsor, be it corporate, government, non profit, or the Holy See.

There is always a specific outcome the sponsor hopes for. The question is, how much pressure the sponsor puts on the researcher not to publish failed or unfavorable outcomes? And, we all know majority of failed research never gets published.

Let's presume that we accept sponsorship. What is acceptable level of pressure? I think this is very subjective.

To be clear, falsifying by modification, omission or addition of data is fraud. Drawing the wrong conclusion is not.

The only time I have seen altruistic, "clean" research was performed by individuals as hobby. Once there are more than one person involved, there will be some preferred outcome. Nature of humanity.

I think there is a useful distinction to be made between the impact that financial support has just by existing, and the leverage that can give the sponsor if they choose to use it.

If I offer a 1mm grant the use of X drug on Y disease, I am being opinionated about where time should be spent and what diseases are "important". This is not neutral, and can be very political.

However, this is in a different universe than if I offer the grant with strings attached like editorial control on the paper(s), or a veto on publishing.

As you note, most failed research doesn't get published but vanishingly small amounts of that are due to outside pressure, it's mostly more mundane reasons.

Research is sponsored by, and done by people, hence it is flawed. But the best of is places value on the rigor of the research regardless of outcome, and certainly does not attempt to silence results that are not favourable to the sponsors goals. Commercially sponsored work should aspire to this in the same way that public research does - the specific outcome the sponsor hopes for is progress in our understanding. In both cases, sometimes the results are damaging to other projects you have (business goals, policy programs, etc.) which is the real test.

Offering a grant on Y disease at all also biases research in favor of pursuing that over other diseases, possibly diseases of more pressing urgency to cure in the realm of improving human quality of life.

I made that point, I thought; but think it is fundamentally a different effect than editorializing or filtering the outputs.

I've definitely seen this in academia. Some institutions will bias in favor of supporting high-visibility research over "meat and potatoes" base-line research (such as reproduction of other experimenters' results, or chasing down un-sexy problems of elaboration or exploration of an information space). Their incentives are to chase grant money and get published, even if the published results aren't thorough or rigorous but will capture column inches (leaving someone else to do the un-sexy work of verifying the claims from a single experiment run).

It leaves gaps in the robustness of human knowledge, a bit of a Swiss-cheese at the edges that isn't necessary, but for the way we reward people for cutting the edge instead of filling the back.

This is why we need government to issue laws to have X money in meat and potatoes research. It was kind of embarrassing with Covid that the last known thorough droplet research was from the 50’s or 60’s as I recall.

> The only time I have seen altruistic, "clean" research was performed by individuals as hobby. Once there are more than one person involved, there will be some preferred outcome. Nature of humanity.

I also think it's worth noting that even though "clean" research does exist coming from individuals, there's also a significant chunk of individuals whose research is even _more_ predetermined for a specific outcome.

Can you name a single occurrence where a publicly funded university in the US or Europe forced a researcher to retract a paper because it conflicted with the universities' commercial interests?

From my time at academia I also don't recall the university or research institute asking to check a manuscript before submitting it to a journal, I'd even go as far as saying that this was unheard of as it would go against the spirit of research.

And in terms of research outcomes the funding bodies usually want publications in high-impact journals, public acknowledgment or results that can be commercialized. There are disciplines where commercial funding plays a larger role (e.g. pharma, medicine), but for most publicly founded research it's a rather bold assumption to claim that the funding is conditioned on achieving a particular result and not on the general impact of the results.

Okay, so what do you call the hiring process at university if not selecting for what kind of papers will be submitted? Mostly selections for good reasons, sometimes less so (nepotism and indeed politics).

Besides, your basic point is not valid: retractions by employer are forcibly demanded commonly in cases of, first, plagiarism, fraud, error and in many cases there are political undertones of this. Granted, in the vast majority of cases it is the researcher using shady research practices because of the researcher's strong political persuasion, but the reverse happens too: universities with (large departments with) strong political opinions that punish researchers or force retractions.

The most famous of these, in my opinion, is the one study claiming vaccines cause autism. Of course, everybody who has family on Facebook is still angry this particular researcher for good reason, and he was definitely grinding a political axe, and well paid to do so:

https://en.wikipedia.org/wiki/Lancet_MMR_autism_fraud

Second, I would argue that a lot of fields of study have "existential problems" that cannot be discussed in papers, or too much in public at all, without consequences for the researchers. What I mean is that there are a bunch of disciplines that have particular vulnerabilities in their foundations, which boils down to famous, extremely authoritative papers that should have been retracted a very long time ago, but weren't. Of course there's easy targets here, such as the Taylor papers [1]. But it goes pretty far: the original autism research was found to have one purpose only: to justify killing children to purify aryan blood lines (research happened right before WW2) [2]. These essentially invalidate a (large part of) a field of study, expose it as a fraud, but they are too enmeshed and refuse to die despite being exposed (and of course, you could more benignly argue they have since expanded and while they don't reject exposed fraudsters like Taylor and -much worse- Asperger, or you could argue they help people, unfortunately for autism treatments research disagrees with that). But you will find them equally in more "pure" research subjects: 99.9% (sorry) of statistics applies only in number series where the law of large numbers applies (which boils down to meaning there is no change in the underlying phenomenon being measured). There's 2 problems with that: first, upon careful inspection it never does, everything is changing. Second, there is an extremely common case that is always excluded: you can measure a "proxy" number for a phenomenon (say, number of theft convictions indicates something about crime rate), but as soon as you make any modification of the source system you are not allowed to use the same measure again, because this act invalidates the law of large numbers by explicitly changing the underlying phenomenon. The problem is that this is a valid criticism of 99% of applied statistics, and we have no resolution to this issue. You could argue that applied statistics works and has positive results, and I would agree. But ... it doesn't make it right.

There are entire fields that are coopted by political attitudes: the medical study of drugs is one example. The relationship between violence and imagining/experiencing violence (famously through TV and/or video games). Just search for long term effects of LSD on Google scholar and you will realise there is more going on than mere mistakes, and it is not really different, but much harder to tell, for most psychiatric drugs, even Valium and Methylphenidate. Many papers that are obviously bullshit are not retracted and some papers that definitely aren't bullshit are forcibly retracted because of the position they defend (such as, for example, that the average use of any given psychoactive medication is fraudulent, and exceed dosage and duration limitations of the medicines, with full knowledge of a psychiatrist who doesn't see a good alternative, which results in extreme damage to patients).

Third there are results that entire field feel they just MUST find, except they never do: the big example is the (direct) physical changes resulting from various psychiatric conditions. Every single study, for decades, has been shown to have been falsified, with a bunch of retractions, but also plenty of unretracted but obviously bullshit research, and several forced retractions of research pointing out the fraud. Just to be clear: this is not talking about depressed patients, say, cutting themselves, and the cuts resulting in physical change, this is about direct physical change (usually in the brain) resulting from (usually) depression. This results in people retracting papers without consequences because all the field sort-of-kind-of-not-really-but-... agrees with the fraud. Again you will find all 3: retracted papers, retracted-but-we're-still-teaching-it-and-rewarding-the-researcher-for-political-reasons papers, and forcibly retracted despite being true papers.

[1] https://www.newyorker.com/magazine/2009/10/12/not-so-fast

"Matthew Stewart points out what Taylor’s enemies and even some of his colleagues pointed out, nearly a century ago: Taylor fudged his data, lied to his clients, and inflated the record of his success."

[2] https://www.nature.com/articles/d41586-018-05112-1

"The Austrian paediatrician Hans Asperger has long been recognized as a pioneer in the study of autism. He was even seen as a hero, saving children with the condition from the Nazi killing programme by emphasizing their intelligence. However, it is now indisputable that Asperger collaborated in the murder of children with disabilities under the Third Reich."

I'm so baffled by this comment.

In what sense is retracting a deeply flawed ("fraudulent") study that was invented for a court case the same as a university pushing a researcher to not publish or retract a paper because it paints the university in a bad light? It seems like you're trying to paint all retractions as equivalently "political", but that's obviously not the case.

This is nonsense. Publically funded research does not have the same issues as commercially funded research, full stop. A properly designed experiment (one capable of receiving public funds from the DOE, for example) will produce useful data regardless of the outcome. Not observing something vs observing something with a well-quantified uncertainty in either case are equally important results.

If a certain experimental result means the experiment is less worthy of publication, then it should not have been funded in the first place. Publically funded research must meet this standard, and in fact receiving funds from all the governmental sources im familiar with have reporting requirements attached.

Good research is interesting regardless of the result. This is the core concept the corporate world fails to grasp.

Yep. Sometimes I see these type of comments online (mostly on libertarian leaning subs in Reddit for example) and the commentators fail to grasp the error behind their reasoning - not everything in this world is a zero sum game.

This is just not true, and sounds like someone talking about a field about which they have no idea. NSF mostly for example doesn't give a shit about your results, all it matters to them you publish something (productivity).

I have worked in both academia and national labs and never heard about funders even caring about my results. Far more prevalent is that funders want your publications to be in high IF journals.

I published several papers while working at Google on Chrome. There is a publication review process that at that time was not onerous at all. In fact, all of my publication submissions were reviewed post facto and basically rubber-stamped. All, with the exception of one, our publication on Spectre, which was carefully vetted to not disclose any sensitive information about further vulnerabilities or violate NDAs with partners. So my experience for publishing at Google, admittedly in a non-controversial area of Programming Languages, was positive.

That said, the corporate culture is bending authoritarian and Orwellian. The new changes are just an further evolution in that arc. This is part of the reason I left in 2019.

When have trillion dollar companies not been authoritarian in protecting and furthering their own corporate interests?

Corporations, companies are some of the last bastions of authoritarianism.

All too true. The biggest challenge Google is struggling through right now is that it's not just impractical, but dangerous for it to continue to behave like a small and scrappy startup.

Behavior patterns that are beneficial for a small company are harmful to a big company. To give a concrete example, "move fast and break things" is fine when you have 100 people relying on you, and should probably be sanctionable (though, in our industry, it is often not outside the context of contracts and pre-arranged agreements) when you have 2 billion people using your system several dozen times a day for critical information, where their lives are materially negatively impacted if that information is false or out-dated.

> To give a concrete example, "move fast and break things" is fine when you have 100 people relying on you,

If you are OK with things to break you are not relying on it, I would say.

> I published several papers while working at Google on Chrome.

I'm curious -- did any of your papers touch on the ethics of browsers or call on specific caucuses of Congress to take action that your employer will not?

And that’s why I was happier when basic research was mostly being carried out in public institutions.

Public institutions tend to observe the same policies, even when there's no direct commercial interest.

For example, if you join a Math department and start publishing papers that cast the department's research interests in a negative light, that's often a good way to get fired.

I mean, seriously, go ask just about any professor of any topic at any university if their field is undervalued or overvalued. They'll almost universally tell you that their field is undervalued, underappreciated, and more important than people think. They deserve more funding, and you should definitely consider majoring in their field.

General proposition may be true, but how can a math publication place the department interests in a negative light? Like "Doesn't matter how hard we try, there are always undecidable facts in our own system"? :p

I still have faith in the research done by hard science research teams.

> how can a math publication place the department interests in a negative light?

With Math, the sensitive topic is utility. Math departments would tend to be upset by someone pushing papers that, say, call attention to the lack of social benefit relative to other fields of research that taxpayers could be funding instead.

I guess a researcher could also spin a narrative about mathematicians' contributions to cryptography, examining the negative consequences enabled by stuff like Bitcoin.

---

> I still have faith in the research done by hard science research teams.

That gets kinda complicated.

I mean, hard-science teams often rely on their expertise, as it's their unique source of value: if the area that they excel in is shown to be sub-optimal, admitting it would mean losing their current career path (and often taking a significant tumble down-the-ladder after transitioning).

So hard-science teams are often largely academically honest in what they do publish, though they're often biased toward casting what they do in a positive light.

For a common example, have you ever met an older computer tech who insists on using a legacy, obsoleted technology because it's what they know best? A lot of academics basically do the same thing.

It sorta sounds like you aren't familiar with how actual math departments function, just by your examples.

My understanding is that the Google is pressuring it's employees (who are research scientists) to refrain from publishing papers that Google management believes casts a negative light on a Google product. It's not theoretical, researchers are quoted in the article and papers have been altered.

I find it difficult to come up with an analogy from a public institution. What is the product the math department would be pressuring it's members to protect?

Sure, each department in a university believes it's work important. That doesn't seem even remotely similar to me as this issue with Google.

What if it's an inaccurate negative light?

The Gebru paper makes a lot of hay about carbon impact, but Google's put a ton of effort into mitigating that, they've been carbon neutral on paper since 2007: https://blog.google/outreach-initiatives/sustainability/our-...

Sorry, I'm having trouble following this comment. In a public institution (as well as at Google) research papers are subject to peer review before publication. Now Google is adding _another review process_ after peer review, I don't believe this is something that would happen at a public university.

Do you have an example of a public university censoring research papers that passed peer review because that university believed the paper cast "an innacurate negative light"... on what, exactly?

The tone of parent comment was "Google should tolerate research that's critical of Google", which I'm sympathetic to.

My rejoinder was that if it's inaccurately critical of Google, like hyping carbon impact without mentioning a decades-long carbon mitigation program, I get a lot more sympathetic to Google's position. Why should they pay someone to spread falsehoods about them?

I don't really care about the process minutiae.

Does it make sense to mention Google's carbon mitigation program in a paper not specifically about Google?

"Large language models can have an outsized carbon emissions impact, but not Google's, those are carbon neutral".

They'd probably want to finess it a little bit more than that, but yeah?

If you're going to bring carbon emissions into it, mentioning current mitigations as well seems relevant unless you're writing an op/ed piece.

> mentioning current mitigations

The "mitigation" in question is buying carbon offsets (I mean there are improvements in DC efficiency also, but those only do so much, and language models ballooning 100x isn't going to be fixed with 10 or 50% efficiency improvements). For the moment "carbon neutrality" is only achieved through the purchase of energy offsets.

That doesn't mitigate. It offsets. Don't get me wrong, still better than nothing, but its not a mitigation.

There's also

* this generation of language models leaning into transfer learning reducing the total number of training runs for different applications

* TPUs being more power efficient than GPUs (the numbers they used in the paper were based on GPUs)

* other energy-centric stuff that's not just offsets, efficiency like you mention in addition to sourcing from renewable

I'm having trouble getting a clear perspective on what's going on. So many of the descriptions are vague and based on apparently informal descriptions, leaving much to the reader's imagination.

> It's not theoretical, researchers are quoted in the article and papers have been altered.

In grad school, PhD students' advisors typically insist on various revisions before publishing a paper, as publications reflect on the advisor and their institution. So there's nothing even slightly weird about Google having the same interest in revising the papers pushed by its researchers.

Unless there IS something weird about what Google's doing? But if so, what is it?

When an graduate school advisor provides feedback on a paper, the goal is to improve the quality of the paper. The peer review process also has the same goal in mind: produce a better paper.

According to this Reuters article, Google's new process happens after peer review and Google's other processes have completed.

"The “sensitive topics” process adds a round of scrutiny to Google’s standard review of papers for pitfalls such as disclosing of trade secrets, eight current and former employees said."

Instead of publishing the paper, Google will now review the paper with an eye to negative impact the paper may have on existing Google product (or lobbying efforts, etc.) Google isn't doing this to improve the quality of the paper, they are doing it to protect their business interests.

"For some projects, Google officials have intervened in later stages. A senior Google manager reviewing a study on content recommendation technology shortly before publication this summer told authors to “take great care to strike a positive tone,” according to internal correspondence read to Reuters."

I think this is very different from the advising process in graduate school.

Preventing people from disclosing trade secrets seem fair to me. Preventing valid research simply because it may negatively impact business strikes me as less reasonable.

"Four staff researchers, including senior scientist Margaret Mitchell, said they believe Google is starting to interfere with crucial studies of potential technology harms."

> I think this is very different from the advising process in graduate school.

All of that sounds normal to me. Including filtering out trade-secrets, which is completely normal when working with trade-secrets in grad school too. Additionally, it's completely normal to filter out intellectual property you might plan to patent; confidential information; proprietary industrial information; information protected by law; dangerous findings (e.g., hackers often omit details of an exploit until the relevant vendor has had time to fix); and a few other categories.

Maintaining a positive, constructive tone is also completely normal. For example, failed experiments are typically described as progressive steps toward an ultimate success; unforeseen problems are discoveries; and major issues are seen as research challenges to be overcome. Or, ya know, stuff like that.

I mean, is that all this story's about? Because if that's it, then it seems like nothing substantial. But if that's the case, why is this in the news?

What is the product the math department would be pressuring it's members to protect?

I’m not sure about math, but in physics it would be string theory which has been a dead end and has mostly served as a welfare program for boomer scientists

> I find it difficult to come up with an analogy from a public institution. What is the product the math department would be pressuring it's members to protect?

couldn't the exact thing that happened at Google happened at a university? For example one researcher could publish a paper criticizing the methods that other researchers in the department have developed because of their carbon impact.

I think it's fair to say that there are internal pressures not to do that - - such a professor would have a hard time thriving in the department if they are attacking the methods of their colleagues.

Mathematicians are pretty open about the fact that their field is unimportant and their research funding is coasting off of winning WWII with Enigma codebreaking. The only reason they deserve money is that they teach mandatory calculus to uninterested non-mathematicians.

>They'll almost universally tell you that their field is undervalued, underappreciated

This is most likely due to an availability bias. They have concrete examples in their own sphere but not near as much detail about competing domains. This leads to the mistaken assumption that their own field more important than it possibly would be if viewed in the overarching context

Oh yeah. Those are surely independent and don’t have aims at all.

This sounds like the typical false equivocation about "all media being lies" when somebody brings up the danger of outright, constant propaganda like OANN.

Just because bias is impossible to avoid, does not mean that all bias is the same.

The goal isn't outright elimination of bias (which is completely impossible), its ensuring incentives that are aligned so bias doesn't become a self-reinforcing feedback loop.

Yes and no. No institution (maybe 1920s Gottingen?) ever matched Bell Labs in their glory days.

> The manager added, “This doesn’t mean we should hide from the real challenges” posed by the software.

If I ever plucked a single sentence out of a long post and called attention to it context-free, I could probably make whoever posted it look fairly bad. But, in the context of news articles, it becomes the headline? Was the context "Try to strike a positive tone...or else you're fired" - or "Try to strike a positive tone...you've shown that these problems are solvable"

It is at will employment, the company makes it clear what you have done there, they want to have a say on it.

If you want independence, goes to academia instead (though it is not easy to obtain it either)

It is at will research funded by google, the researcher makes it clear what their findings were, they want the original research paper to have its say based on results not spin.

If google wants specific outcomes from research, go to the marketing department instead of funding real research.

You are offering a false choice / false dilemma:

https://en.wikipedia.org/wiki/False_dilemma

Many of the comments on this thread are expressing concern and (more or less) asking 'why does it work like this?' and (in some cases) 'it would be better if it didn't work like this."

Don't all corporations want people working for them to be upbeat rather than calling cars death machines etc?

Gebru was fired for unprofessional behavior, not because of her research.

Jeff Dean’s letter with details: https://docs.google.com/document/u/0/d/1f2kYWDXwhzYnq8ebVtuk...

That can also be true.

Google is a for profit corporation, not a college campus, news at 11.

People will act like this is up for debate.

Everybody knows that. You can think it's a bad thing for a private company to do bad things, even if they're free to. Stating a fact isn't an argument.

I think the comment flies more in the wierd way Google is treated by the media when other corpos aren't.

E.g. noone is batting an eye when Apple chooses cherry-picked based datapoints for their performance comparisons or when Stanfords research department spins a paper to get them more funding.

For Google we keep getting this streams of "Google is acting like any other corporation, OH GODS". It's... wierd.

I dont know if a case where Apple hired ethicists and then fired them for their research? Admittedly apple also doses all their employees with NDAs up the wazoo so we simply can’t know these things.

All I'm asking for is the freedom of tenure combined with the pay and equity of industry.

The top dogs have both, with multiple affiliations.

therefore we're supposed to ignore their transgressions, since they're to be expected?

Getting a job at Google and expecting academic freedom is certainly a pipe dream.

Yes, I agree it's not surprising that this happened. Doesn't mean we should stop paying attention or caring.

Yes, and that is ok.

But then nobody should call whatever they are publishing "paper". Because if you "research" with a PR-directive, it is literally not science. If they want to "research" like a corporation, kick them out of Google Scholar and scientific discourse, right now, please.

Propaganda disguised as research has historically been a source of immense harm to the public good, and identifying it is a crucial step to reversing that harm.

This whole saga is such a PR “own goal” by Google. The vast majority of corporate research labs just don’t fund research doubting their own ethics in the first place. Company leadership should choose an ethical stance and defend it. It’s an abdication of responsibility to say, well the ethics are an open question, we’re going to fund some research into it. And now Google is being punished for their abdication of responsibility. If they had simply established a corporate policy that said, we believe facial recognition is good when XYZ conditions are met, and defended it, they wouldn’t be in this situation.

A lot of the stuff really is an open question though.

For example if you’re training models on historical data to determine credit worthiness, you can end up with models that penalize people based on race, or zip code (as a proxy for race), or names (same thing again).

How to correct for this is not obvious, removing the fields from the data usually just has it appear in some other way that’s correlated to race.

My understanding of current approach is to leave the fields in and then check to see how out of whack things are and there are a few approaches to try and correct for this (also an open question).

The reason modeling risk on race is unfair is because while the color of your skin may be strongly correlated with being a bad (or good) credit risk due to a whole host of complex historical reasons that lead to bias in the data, it’s not because of someone’s skin color that they should be considered a bad (or good) credit risk. You want the models to be judging based on more causal details. Not data that’s biased due to historical favoritism or exclusion.

Doing this poorly can further cement the pattern in the data and make it worse over time in addition to just being less accurate.

That said, you want people working on this looking for pragmatic solutions and not just promoting their own agenda while spinning things to seem worse than they are with emotionally loaded language targeting the company. Making demands and then going public on Twitter about your failed ultimatum doesn’t inspire much confidence.

Seems reasonable for a company to not fund research that directly goes against their bottom line.

When the result is predefined it's not research anymore, it's advertisement. Still calling it “research” is fraud.

Exactly. This isn't science. Sometmies science finds uncomfortable results, and if it's not truthful about them, then it's not science.

This. Google wants their cake + to eat it to by hiring the best in class Ph.D's with the allure of doing "academic research" but paid a big-tech salary.

Turns out it's mostly PR and buying yourself credibility at this point.

Honestly find it confusing how almost everyone from the outside could see that teams reason for existing was PR but the team itself seemed totally oblivious to the fact.

Upton Sinclair comes to mind: It is difficult to get a man to understand something, when his salary depends upon his not understanding it!

You think it's impossible management factions had different agendas? Or researchers didn't expect to spend their whole careers there?

> When the result is predefined it's not research anymore, it's advertisement. Still calling it “research” is fraud.

Like, when you see someone who's an amazing software engineer with a great salary, and you know they spend a lot of time doing research. Then you suppose the research must be paying off. Over time, survival tends to equal truth.

For almost any paper I read, before I pick it up, I know there were statistically significant findings. Those results were not predefined. Instead they are part of the academic review process that filters out null results. That same process (depending on the field) also rewards novelty or all sorts of other things. It's very much a human process.

I'd guess it's an operation of: here's a set of acceptable results. P-hack your data to show statistical significance for at least one of them

If Google did not want to fund research that conflicted with their bottom line, they could have avoided hiring researchers interested in that kind of work. Instead they hired these people and told them they would be free of censorship. Now that these people work there and depend on Google for their paycheck, Google is instituting this new policy. During a global pandemic.

Sounds like censorship to me: hire the people in the field who might publish research you disagree with, then pressure them from a position of power.

You're confusing the ideas with their presentation. Google never allowed someone to post a paper under their name saying ‘datacentres generate CO₂, that's evil and we should focus on smaller datacentres’, but that doesn't mean they didn't care about CO₂ production; clearly they did, since they're now carbon negative.

Same thing here. The researchers' jobs aren't to publically deface Google, they're to help direct Google down the right path.

I agree that Google may have been interested in hiring these researchers to help "direct Google down the right path". That seems reasonable to me.

I'm not seeing where I confuse "the ideas with their presentation" or where anyone talked about researchers "publicly defacing Google". The Reuters article is talking about ML and AI researchers who are now (recently) finding their research subject to a new review policy, above and beyond the typical peer review that they expected.

What's Google's going to care about is "are we going down the most profitable path" though.

It's a bit different from the right path

Google always had internal review processes. If your argument is not that the new review process is unreasonable, just that people were hired under the expectation that there would be none, then I get where you're coming from but it doesn't sound very likely to me.

> Google always had internal review processes. If your argument is not that the new review process is unreasonable, just that people were hired under the expectation that there would be none, then I get where you're coming from but it doesn't sound very likely to me.

Here is my steelman position: I think the objection is that the old internal review was reasonable, and while the argument can be made that the new one does not seem unreasonable to many people under the principle of "don't bite the hand that feeds you", it's reasonableness in a faux-academic setting is at least debatable, and in any case it is inarguable that the new one is more restrictive than the one that was in place when these researchers were hired.

So, at the very least, this seems like a unilateral bait-and-switch on Google's part. The company is now reneging on the representations made to prospective employees regarding their academic freedom.

"I have altered the deal; pray I do not alter it any further.": https://youtu.be/WpE_xMRiCLE

In twitter culture, public defacing seems to be the primary way to help direct things go down the right path. Want to get google down the right path? Publicly call them out for not doing it already and get enough other people to do the same that there’s no option but to loudly comply.

> and told them they would be free of censorship

Of course they never really said it for simple legal reasons, when you sign the employment contract you actually agree to a lot of censorship.

We already know that ,,self regulation'' doesn't work. It made sense to me that when I was working at Google I didn't talk bad about the company that I was getting my paycheck from.

Alphabet is still a great company, but that doesn't mean that it doesn't need checks and balances from outside.

> I didn't talk bad about the company that I was getting my paycheck from

Have you ever been in a union workplace?

> Have you ever been in a union workplace?

why would he want to be in a union when he is a top earner compared to great majority of people in the same country? and aside from a relative freedom of speech he works in very favourable conditions with a lot of perks and a generous severance package if he gets fired? working at google is the very definition of privilege.

Maybe then he would not have to tow every pr line in fear of being dismissed

Heaven help him, though, if he says anything critical of the union. Tire slashing and other forms of intimidation routinely made newspaper headlines in UAW territory in the era before the Big 3 automakers collapsed. At least dismissal doesn't involve people following you to your home.

It isn't businesses that make US unions look bad, they did it to themselves.

What other balance to corporate power do workers have?

Unions are for more than just getting higher pay.

If Google (perish the thought) decided to pull some shady reviewing stunts and make your work life so intolerable that you felt your only option was to force the issue and threaten to resign - being part of a union could have helped with that.

HR is a bunch of people who represent the company's interests in negotiations and transactions. Unions are the same thing for employees.

We've been told, repeatedly, that unions are socialist organisations that want to destroy capitalisim and bankrupt employers. This isn't true.

> union when he is a top earner compared to great majority of people in the same country

Why does that make creating a union/engineer council less desirable? It seems good to have a democratically elected representative of Google engineers who can talk to management, even if those engineers don't feel like negotiating on wage stuff, they still have a real avenue to air concerns.

> why would he want to be in a union when he is a top earner compared to great majority of people in the same country?

Didn't Google get busted fixing wages along with other tech companies?

https://en.wikipedia.org/wiki/High-Tech_Employee_Antitrust_L...

No, but my work life was amazing.

There are important things to address at Google, like not having enough sitting toilets for men (there's 3 times as much for women even though there are much less women in the office where I was working) that an union may help with, but that's different from the product PR.

Forcing men to poop standing up is probably some productivity hack.

My first inclination was just to move on but you didn’t post on a throwaway so I will take the bait.

Why do you think the toilet issue you described is a problem that needs to be fixed? My experience in every corner of life in America to date has been separated bathrooms by sex and the men’s restroom containing far fewer sitting toilets. We all know this is because women sit for both natural duties and men, generally in the US at least, only sit for one. This is so basic I assume it’s not what you are talking about but please correct me if not so we can engage on that front.

What I assume you mean is Google is 90% male and 10% female (no idea what it really is) and yet the bathrooms are 50% male and 50% female. You specifically mention sitting toilets which is what I am having trouble understanding the relevance of.

Are men regularly forced to stand in line instead of being able to walk right in and go? If so that sounds like a facilities growth issue and not a sex imbalance issue. You could propose turning some female restrooms into male but generally restrooms are placed far enough apart from one another that there would have to be 0 females in the area serviced by those restrooms for that to work. The presence of even one would require facilities assigned to females or mixed gender restrooms (a different argument I fully support).

That leaves your only other option as I see it for the female restrooms to be smaller than the male restrooms which is just ridiculous. No one would build the building with that hopefully temporary imbalance in mind and you can’t just repurpose half a bathroom as long as the sexes are being separated.

I will bait too, as someone who has stood in line for male toilets in a Google office.

In the office I was in there where generally 1 sitting and 2 standing seats per male toilet.

The standing one are quick in / quick out and almost always free. But good luck finding vacant sitting spot after lunch.

Edit: female restrooms generally had 3 sitting spots, so same size.

> Why do you think the toilet issue you described is a problem that needs to be fixed?

Because this way Google was effectively paying me loads of money to just go through 7 floors of bathrooms to find an empty one when I needed to use it. It's a waste of corporate resources to use a human resource's time like this. At the end the solution for me was just using the disabled toilet, as we didn't have disabled people on the floor at all.

One solution would be to just build more sitting toilets instead of the standing ones, as those ones weren't used as much. But on 1 floor there were 2 bathrooms for women and 2 for men, so yes, repurposing some would have helped.

It‘s mind boggling that one needs to explain this (especially in a community like hn). Seems like literacy in basic economic principles was replaced by feelings in many people

Did you miss this part?

"If so that sounds like a facilities growth issue and not a sex imbalance issue"

To make this into a gender issue, you have to argue google can't afford toilets

Of course it’s a growth issue, Google can’t easily afford the prime real estate with the current housing prices. Load balancing is just a temporary measure that can be done.

Google tear down meeting rooms to have more space for cranking more people in the same office.

Luckily I was working there in the good old days mostly.

Are you seriously arguing that one of the largest and most profitable megacorps in the world can't afford enough toilets, so we'ce gotta take a toilet away from female employees?

Profits aside, we do not consider stopping cramming people in an office, or hiring in a more agfordable location? What happened to common sence and self respect? This the literally shittiest, most submissive argument I've come across. If it weren't a civilised forum, i would've colled it bootlicking

I’m in favor of a more modern arrangement, where all restrooms are for a single person and can be used by anybody. It works well no matter what the gender ratio of the workforce is. Eventually we will all wonder why the heck we did it the old way.

It's pretty obvious why -- people prefer to spend less money.

Some floors of certain buildings have that arrangement, but because single occupancy restrooms take up more floor space, there tend to be fewer of those, so the capacity might not work out as well as imagined.

Personally I find it more helpful to adjust my biological clock to avoid doing my business at the same time as everyone else.

Most of the time, believe it or not, details like toilets are outside Google's control, let alone unions'. I heard so many stories when I was in the NY office, like facilities not being allowed to do anything about the big gaps along the stall doors in the tenth floor bathrooms until they bought the building for $2B and everything was torn down. Or the two years it took to get approval to place the stupid logos on the outside. They weren't even lighted, which would have definitely added extra years. Every month it was a different person's turn at the city to reject the application, often for something that had been okay the previous time. There's hell, and then there's the NYC DOB.

The bathroom situation stank (pun not intended), but frankly there are other hills to die on.

Can you elaborate? Who are "we", does this refer to some scientific community studying these things?

In my opinion, this is different: Google is hiring research scientists and telling them that Google would like to pay for their research. When hired, Googled told these people that they would be free to conduct research in their field without this kind of censorship. Now they are being censored, this is a new policy.

In my opinion, this is very different from a typical employee "talking bad" about their employer.

Any researcher who thought they could publish research about 'race and gender' and not have it reviewed by a corporate sponsor is incredibly naive.

It's an extensively sensitive subject and particularly in a field with such vague and 'subject to interpretation' results, the sponsor corp. is going to want to have a look. Considerably more so if the research is about the institution itself.

There was a specific case recently where the researcher criticized Google for something, but Google asked the researcher to include truthful information about the changes the company has already made to the system in question wherein it's been improved etc. This is a perfectly fair request by the sponsor, I think.

It's also naive for us to consider that what people chose to research, how they approach it, the results they are looking for isn't somehow altered by the researchers own biases. In fact, it's hugely arrogant for researchers to believe that they have some special access to the truth in this regard.

Perhaps researchers who accepted these positions with Google were naive. That is certainly possible but it can't be argued: Google hired these people under the pretense of letting them do their research free of censorship and, now that they work for Google, this censorship is being imposed.

I'm not sure what the rest of this comment is alluding to... The posted article from Reuters doesn't mention race and gender, the article implies these researchers focus on AI and ML.

In regards to the "specific case", I would have to guess at what that case might be. Were I to guess, it seems like I'd be guessing wrong as this assertion seems to lack grounding in fact.

Naive or not, peer review is in place and part of it's purpose is to keep the biases of researchers in place. History has proven that this is far from infallible, but it's the process we have today and the process Google agreed to when it hired these researchers.

Here is my theory: these researchers were not naive and they knew the deal, they could work on their research as long as the topics were not too sensitive and as long as the results did now harm Google's image too badly. So what likely happened (imho) was either a miscommunication, or they got results they did not expect. Either they did not realize which topics are too sensitive to touch, or they believed they'd get different results and did not.

I cannot entertain the idea that they signed up with Google believing truly they could have full scholastic freedom. I am pretty sure it is even covered in elementary ethics classes that such a scenario is itself questionable.

They probably felt that they had 'scholastic freedom' and they generally do.

But if your job is 'AI Research' and you publish a piece about how much electricity Google uses to 'train models' ... well, you can expect scrutiny.

In this case, the researcher was simply asked to hold the paper until more information was included about the fact that Google has actually improved certain paramaters quite a lot since the time the data was referenced in the report.

This is extremely fair from Google.

If the research was purely concerning some AI results, even how they could be unjust - fine. But as soon as your research specifically includes the sponsor, then it's perfectly fair for the sponsor to add more information.

It's not censorship, in that case, it's frankly the opposite.

The 'miscommunication' here is a kind of naive thinking by researchers who probably have never had normal jobs, who have never worn a 'company hat' and may not have any internalized understanding for communications.

For many researchers - money comes from magical taxpayer funds, and the 'Uni Model' is the only model they understand and some might think that's how the world ought to be.

If Google was giving people cancer due to the tar in their products, the researchers were discovering this - and Google was trying to suppress this, then you'd have a serious problem.

This is not Marlborough/Cargill/Exxon, it's Google and it's an entirely different situation.

The 'hammers' used by the social justice fronts from the 1960s just don't work anymore, because we have made progress and nothing is obvious.

>Alphabet is still a great company

To those it pays, yes...

Poor Google managers. Nothing they do, say and think is private anymore. How sad :)

If only there was a strong privacy culture there.

The directive to strike a positive tone could be nefarious or benign; we can’t tell from the info in the article. Every research paper has a tone that depends on the personality of the author - I’ve seen it even in my dry field of quantum optics. Every result has many implications and how you discuss them, which implications you emphasize, and the language you use will affect the tone, even if you are not explicitly furthering an agenda. A given article could have a range of legitimate tones, depending on who wrote it. Asking a researcher to be on the positive end of the legitimate spectrum is benign. It can of course go to far.

Consider the historical parallel to the tobacco industry’s research departments. And asbestos. And lead. And talc. And etc. The researchers found that the products were harmful, but their results were squelched. “Do no harm,” indeed.

Well, Google removed "Don't be evil" as their slogan quite some time ago. Don't think that was by an error.

This is a rumor that won’t die because it’s too enticing to believe and tack onto any anti-Google sentiment.

The phrase was only moved from the introduction to the conclusion of the code of conduct, never removed.

Checks out, as of September 2020: https://abc.xyz/investor/other/google-code-of-conduct/

> No Retaliation

> Google prohibits retaliation against any worker here at Google who reports or participates in an investigation of a possible violation of our Code, policies, or the law. If you believe you are being retaliated against, please contact Ethics & Compliance.

Well, it is safe to say this is hot garbage then.

I would love Google to remove all the self entitled BS from this statement, so they can act as an everyday faceless big company which is anyway what it is doing, not some moral beacon for the mass to look up to.

The statement exists only to generate traffic for the news outlet for spinning.

But they don’t emphasize it or mention it officially during orientation. It was only mentioned during one of the smaller tech trainings facilitated by a longtime Googler. To paraphrase their words, “I’m not sure why we don’t say this anymore.”

So the statement that it’s really not a slogan anymore is quite true from my experience.

I was at Google (off and on) from 2007-2019. The company has definitely backed away from it both officially and unofficially.

Correct.

Though they did replace it with the motto "do the right thing". Which to me is even more scary, as it means that Google decided to stop being a neutral entity within the boundaries of not causing harm, and decided to become a political actor that is committed on advancing its own idea of "right". What if (as it happens) my idea of what is right doesn't match that of a 1.7 trillion company that controls much of the information flow worldwide and provides most of the services I use in my daily life?

Obviously your comment will be downvoted to oblivion without our friends here in the HN audience explaining why, but I have precisely the same experience.

Please can we stop repeating this at every opportunity? I'm sure everyone has heard it a million times. Whatever truth and insight it once conveyed has rather faded away.

yeah. and it’s not like “don’t be evil” is something to be proud of. it’s the bare minimum to be expected

Am I the only one that gets tied of the "Don't be evil" post each time a Google related post is made? It's like beating a dead horse.

Maybe because the horse is still sitting there dead? :)

Pretty sure the Devil's lawyers asked for that.

"The erudite Faust is highly successful yet dissatisfied with his life, which leads him to make a pact with the Devil at a crossroads, exchanging his soul for unlimited knowledge and worldly [power]."

tl;dr - if we don't create it, control it, and make sure everyone knows about it, someone else is going to make all those decisions for you. Its coming regardless

These are not equivalent comparisons to me. Tobacco was always harmful, inhaling smoke in general is just bad, and the others could be through improper use but safe otherwise. Let alone all of what you mentioned will not have the impact AI will

AI's potential is equally good and equally bad and any of the bad is purely because someone or some group employs it that way.

The real danger of AI isn't the obvious tinpot dictators using it drive war but in politicians and government officials hiding behind the term as an excuse for their bad decisions. The more you demand of your government to give you stuff or services the more likely you will be subject to this tech

I say this each time the subject of AI abuse, facial recognition or similar comes up.

This is not something that cancel culture will stop. Nothing is going to stop it from coming. So the best alternative to the technically and morally (hate using that word) driven people is to hit it head on from all directions. This means working to make sure it does what it is supposed to do but also that everyone knows what it really cannot do, working the legal end to insure there are laws and consequences for employing the technology incorrectly, and through the information end in keeping the public fully aware of who is using, how they use it, and how to legally protect yourself from it.

> someone else is going to make all those decisions for you

Well, yes, and in this case that someone appears to be Google, which is why people are complaining about them making AI decisions.

It seems to be an unfortunate reality of doing research for anything other than the sake of research.

My physics teacher worked for some kind of government air pollution modelling group in the past, and effectively said that they would often be told "Here's the result, find us data to make it work".

It seems like corruption that does billions in damages and costs lives

"The basic rule of Government, never set up an inquiry or research unless you know in advance what its findings will be,"

- Sir Humphrey

Wasn’t the requirement to add a mention of the things google has done to improve the situation, rather than a squelching?

It’s like if an asbestos company did a bunch of research on healthy insulators, then found that the original products were in fact harmful. Before publishing it, the bosses demand mention of the prior work towards healthy insulators.

If the outcomes of research don’t align to Google’s business strategy, they’re going to get rid of the research/researchers.

Previously, ethics was good PR; now, it’s a risk in antitrust cases. This will continue to ramp up.

It will also continue to ramp up even more as growth dries up. The internet can only get so big, especially on a quarterly basis, and these companies are already very big. They're public and pressure to grow is huge.

It's going to be a rough ride throughout the next decade.

You may well be right that this is Google’s current thinking, but it doesn’t look like a winning strategy to me. Once more, they are on the defensive trying to offer flailing justifications for an opportunistic initial move. Even if you don’t care about the ethics, or are on their side, I think you’d have to question their execution.

Oh, I don't think these are post hoc justifications. Much more likely Gebru was intentionally forced out - I wouldn't be surprised if Google leadership basically said "she's a loose cannon - move her out in the next 6 months" and there was just no good way.

No. More like, researchers can do as they please, but if they're going to attack Google or be negative about something related to Google, then there's going to be a review and words will have be chosen carefully. There's a big difference.

It's important to draw a distinction between positive tonality and censoring negative results.

For example, the recent COVID-19 vaccines: they're commonly reported to be 95% effective and have some risk of side-effects. This can be reported factually, or spun in a positive or negative light. For example, a sensationalist might try to stir up interest by spinning a narrative that focuses on the vaccines being unreliable and dangerous, as proven by science that Big Pharma doesn't want you to know about!

And as we've seen, sensationalists can get cult followings. However bad their ploy may be for society overall, the sensationalist themself may stand to profit.

So as reprehensible as censoring legitimate scientific research would be, a directive to "strike a positive tone" sounds like it could be a very different, far more sensible thing.

> And as we've seen, sensationalists can get cult followings.

We're talking about published research papers, not people posting stuff on twitter. I have not seen anyone get a "cult following" from a research paper.

It's common for researchers to publish a journal article in a more academic tone, then use it as a basis for legitimacy in public venues.

This happens at both individual and institutional levels. Examples:

1.) At an individual level, someone might publish a study on a social issue that's important to them, adhering to relevant academic standards, then establish a popular identity as the author of that study. Afterwards they can comment on blogs, in media releases, on talk shows, etc., using the perceived credibility established through academic publication to pursue their personal goals.

2.) At an institutional level, someone might publish a study on some issue relevant to an industry, then use it in political lobbying related to that industry as well as popular press to build good-will. Many academics have industrial sponsors for this reason.

Well, I'd be fine with Google putting its thumb on the scale in terms of how researchers engage in public press conferences/media releases/talk shows, etc.

What we're discussing, however, is "striking a positive tone" in actual research papers.

> What we're discussing, however, is "striking a positive tone" in actual research papers.

Yeah.. that's what pretty much every academic does. Most talk up the merits of their work and the general importance of their field. The idea's to be constructive, always looking for avenues of improvement. For example, if you test a new medical treatment and it's not great, you don't trash it as a failure but rather focus on how it can be improved to get more positive results.

Though I'm starting to get the impression that you're reading "striking a positive tone" as meaning something other than what I'm seeing in those words. So to avoid talking past each other, what exactly are you seeing? And do you find it somehow problematic?

Allow me to introduce Andrew Wakefield. The father of the antivax movement.

That's a good counterexample, although I'll note that Andrew Wakefield is referred to as the founder of the "science by press conference" phenomenon - so not really his research paper that was the single thing that attracted attention.

Hah, reminds me of this literature teacher I had in school. She asked us to write a report on the positive aspects of Hamlet. I remember sitting at home and thinking "Well... It could have been worse - more people could have died, that's positive thinking, right?"

> She asked us to write a report on the positive aspects of Hamlet.

Spoiler Warning!

Rosencrantz and Guildenstern get what is probably their just desserts.

If that’s too uncertain given they may not have known the fate the King had in store for Hamlet, then Fortinbras.

Fortinbras is a pretty standup guy through it all and his closing lines provide the possibility of a better future.

The “sensitive topics” extra review seems completely fair and obviously necessary. Anyone acting like that amounts to censorship in a private company is totally off their rocker.

This article seems really weak, and of course there’s the obligatory attempt to tie it in with Timnit Gebru’s resignation and her very inappropriate behavior, even though for all of Google’s flaws, Google clearly and obviously did the right thing both to disapprove Gebru’s weak paper and accept her resignation based on her ultimatum.

"Gebru’s weak paper"

Could you please back up your claim as to the paper's quality - do yoy have credentials and background that enable you to cast this judgement, or is this pedestrian?

I don't need credentials to read a paper which muses about current events.

You want to outsource your thinking to the credentialed? Go ahead but the rest of us don't have to. The paper is out there and it's quite easy to read, because it's not saying a whole lot.

It has leaked. Activism masquerading as science. Yes, people also call critical race theory and intersectional theory academic disciplines. They hold conferences, have titles and credentials, are professors, etc. So yes the paper is high quality for a certain niche.

It's not a technical paper for not a technical conference. It's a "raising awareness" paper on identity politics beating the same drum, without much novel insight or analysis or new data. But yes, it has 120 or 200 citations, a fact that was proudly highlighted by authors on Twitter to underline the scholarly quality.

Yes, gender seminars and whiteness workshops are also properly credentialed.

Theology is also a discipline but perhaps not the same as sciences.

Quality is indeed relative.

Can you share a link to it? I’ve followed the situation but not seen the paper itself yet. Very interested to read it and judge for myself.

https://www.docdroid.net/HZvDVwN/sp1-pdf or https://bunkerchan.xyz/.media/de21413543c09ac6d02aa5eeb08424...

Sure, I wrote a lot about this already when the paper leaked,

https://news.ycombinator.com/item?id=25314824

The full paper is also out there, circulating on various file upload sites. Your points stand even to the full text.

The problem is that Google wanted to monetize AI via marketing itself to a higher share price and more panache (sell the sizzle). If AI were solely an internal tool at Google, none, or very few, of these ancillary problems would have presented themselves.

Or, they could have just labelled all these efforts with the less sexy and scary moniker of "machine learning".

It is almost 2021 and the only stuff ai has “solved” is annoying customer support chat bots, endless youtube videos and funny deep fakes. Just like IE it will be 2022 by the time ai will predict covid and 2030 by the time it will find something useful. What happened to them 10 per day ai startup posts in here?

Interesting opsec: “according to internal correspondence read to Reuters”. Seems like people are afraid of a fwd paper trail.

It is common for university professors to be advisors or C level execs in startups while they still hold academic positions (specially in AI), for me that's many times more concerning.

Idea: AI bot for detecting positive tones in Google's papers.

How does it make you money?

My money-making idea from this is AI to detect sentences or fragments from your emails that could be taken out of context and made into embarrassing headlines if the email gets leaked.

Then, is academia a good place to go to for research instead? If not, what should researchers do?

No person or company or country sets out to be evil. It happens little by little.

Response to the paper arguing many ML models are very underspecified?

tl;dr;:

+ "Google’s new review procedure asks that researchers consult with legal, policy and public relations teams before pursuing topics such as face and sentiment analysis and categorizations of race, gender or political affiliation, according to internal webpages explaining the policy."

+ "A senior Google manager reviewing a study on content recommendation technology shortly before publication this summer told authors to 'take great care to strike a positive tone,' according to internal correspondence read to Reuters."

+ surprisingly, there's not much else in the article beyond those two sentences

Edit: as commenters noted, there is more in the article. I apologize for not being clear that the last bullet is just MY OPINION. That is, for me the rest is not as meaningful as some consider it. I guess, if you care deeply on the subject, you'll want to read the article and decide for yourself.

Why does your last bullet point make me feel like we read two different articles?

fwiw I agree with the parent comment. This article, especially the ludicrous attempt to tie in the Gebru stuff, is super weak and absent of any details or anything substantial.

I don’t see anything in the article that suggests Google suppresses unfavorable research or censors authors whatsoever. Instead it sounds like a lot of whining and complaining from exceedingly privileged researchers who want some version of tenure to apply within for-profit companies and seem too immature to accept that legal and compliance functions within a large corporation very differently than in academia.

Why did you find the mention of Timnit Gebru ludicrous? The issues are closely related. In fact, I was _expecting_ the article to mention it.

Google tells Researcher 1 to re-word things in X way for Y reason.

Google tells Timnit Gebru to re-word things in X way for Y reason.

The X and Y are the same.

Based on the rest of your response, I believe you are having trouble reading the article in an objective manner. You say that it sounds like whining to you. That’s because you already think it is. You came into the article with this viewpoint about academics in corporations already established. This is why I also found your comment hilarious. You already think Timnit is a whiny researcher. You say this article is about whiny researchers. Yet it’s _ludicrous_ to mention Timnit in this article?

Also, Jeff Dean lied in his email. Maybe that’s why you’re confused about the Timnit situation. I’m assuming you don’t work at Google.

You appear to be mistaken on nearly all the facts about the Gebru situation. Given this it seems clear we would draw very different conclusions. Anyone claiming Jeff Dean lied must be so disconnected from the actual facts and so prejudiced to blame Google and view Gebru’s actions as acceptable that their opinion amounts to nothing but partisan gainsaying.

To be clear, it’s ok to disagree with Jeff Dean or to dispute his charitability in accepting Gebru’s ultimatum-based resignation, but to claim he publicly lied is beyond the pale.

It really isn't. He lied. Normalize calling out corporate executives when they lie.

I wish he hadn't. I'm disappointed that he did. But I don't know how else to describe sharing statements one knows (or should know) to be false.

No, he really, really didn’t lie. I wish people predisposed to only see Google as being on the wrong side of it would stop saying stuff like this.

I'm a Google employee who, I think most would claim is generally on Google's side of things.

He lied. There are falsehoods that he knew (or had a responsibility to know) were false in a public statement he put his name on. They've been repeatedly corrected both by other employees and by data.

Specifically, he at least lied about a 2 week requirement for paper reviews in the internal publication approval process. While that's listed as a further requirement, they don't "normally" require two weeks.

(I wouldn't, for example call the resignation statements lies. Those were unintentional falsehoods)

I doubt we will come to any common ground here. If you think that Timnit Gebru did not resign, it’s probably not possible for us to convince each other of anything, given that I view that as a core, irrefutable fact bolstered by tons of corroborating accounts. The idea that whether it was or wasn’t a resignation (it irrefutably was) can even be called into question whatsoever is completely incompatible with the facts. That attempt to subvert reality was retrofitted onto the situation later, once the consequences of her inappropriate actions had set in for Gebru.

I'll just point out that Google itself has backed off on statements that she resigned. Sundar's email (the most recent public comms) limit itself to describing it as "her departure", and my understanding of California law is that her "departure" would be classified as a dismissal, not a resignation, though ultimately this depends on very particular specifics that aren't public, such as when her last paydate was, however what I've seen seems to suggest that it would lean towards dismissal. [0]

For what its worth, I don't think "threatening to resign" is the same as resignation. Much as I don't think "threatening to shoot someone" is attempted homicide.

[0]: https://www.cnbc.com/2020/12/09/google-ceo-sundar-pichai-mem...

That comment from Sundar bolsters the fact she resigned. People were raking him over the coals to reverse the earlier claims of resignation to say she was fired, and seeing the CEO refuse to do that is pretty huge evidence of how secure the legal team must feel that calling it a resignation is completely correct and needs no retraction or correction.

I don't see how the CEO refusing to continue to say she resigned bolsters the claim she resigned.

> calling it a resignation is completely correct and needs no retraction or correction.

Alternatively: publicly calling it a firing is essentially an admission of guilt. That's the one thing Google's lawyers would insist Google cannot do.

Presumably neither one of us are the experts on EEOC and employment litigation strategies. So instead I'm going to take the statements at face value. And at face value: Google is no longer willing to state that Dr.Gebru resigned. Anything else is spin.

The CEO didn’t refuse to say she resigned. He refused to say she was fired and refused to contradict earlier statements of her resignation.

> He refused to say she was fired

I agree! He also refused to say she resigned. Fancy that!

The important thing to note is that one of these was a change in behavior and one was not.

It’s disgusting that you literally have two Google employees providing you the truth and you just keep saying “No, you must be mistaken.” I have eyes, I can read. I have a brain that functions very well. A lie is a lie is a lie. And even if lawyers some how manage to button this up, it’s still a lie in the court of public opinion...

EDIT: Sorry, I understand that you don’t have as much information as us and you’re just going off based on tweets and stuff. But come on. It’s insulting to be lied to like that. I hope you can understand that feeling at least.

I think (know for a fact) they were lies also. There was no huge list of demands.

Yes, he literally lied by omission. Internally, there’s an entire data analysis available for viewing. We have literally asked senior leadership why his email does not align with facts and we have not gotten a response. Internally, you can also read the exact feedback that Timnit’s paper received. Like I said, I’m assuming you don’t work at Google because even if you didn’t like Timnit and just think she’s a whiny researcher, there’s no way possible that you could say that Jeff Dean didn’t lie.

Besides the 2 week thing, he also misrepresented the “list of demands” to make it seem like it was worse than it actually was. So much so that I’m even surprised he was allowed to publish the email publicly.

I can’t imagine a single lawyer looked at his email, honestly.

Hypothetical but likely scenario below.

Jeff Dean: “Oh, you say she made a list of demands. Ok, let me put that in the email. Oh, you said there’s this 2 week deadline that she didn’t follow. Ok, let me put that in the email. What else do you want me to add?”

I believe the question you should be asking is _who_ told him these things. Please guess.

What about this:

"Subsequent correspondence from a researcher to reviewers shows authors “updated to remove all references to Google products.” A draft seen by Reuters had mentioned Google-owned YouTube"

or this:

"""A draft reviewed by Reuters included “concerns” that this technology can promote “disinformation, discriminatory or otherwise unfair results” and “insufficient diversity of content,” as well as lead to “political polarization.”

The final publication instead says the systems can promote “accurate information, fairness, and diversity of content.”"""

or this:

"A paper this month on AI for understanding a foreign language softened a reference to how the Google Translate product was making mistakes following a request from company reviewers, a source said"

or this:

"A draft described how such disclosures could infringe copyrights or violate European privacy law, a person familiar with the matter said. Following company reviews, authors removed the legal risks, and Google published the paper"

Seems like there's a lot more than "two sentences".

> internal correspondence read to Reuters

so, all of this opinion piece is based on hearsay?

How do you think journalists get stories like this if not from insider sources?

journalist should at the very least receive, see and validate documents veridicity. here it's "internal correspondence read to Reuters."

from a highly politicized topic with a lot of disinformation being thrown around from both side, it's hardly sufficient.

but apparently you all happy enough to live in a post factual society where journalist have no longer to validate sources' claims, so be it

Lol, this is how journalism has been done for decades - There is nothing 'post factual' about it.

yeah no, imagine watergate without the address book, the tapes and the bank records.

Oh wow. Google has become Evil.

Isn't this exactly what "Don't Be Evil" was supposed to ward against?

Well, "Don't Be Evil" was a very successful PR slogan. "Framing in the positive" in another context.

Once the corporation grows beyond Dunbar's number, it probably begins to look and act just like every other corporation...

This comment is getting old now...

2021 will be an interesting year and the feedback I got after my latest booklet / lit review aimed at Silicon Valley problems is just explosive: tightening regulation, legal actions, anti-monopolies concerted effort will ramp up dramatically at worldwide level.

The whole topic around bias in AI is overblown. If anything is to come out to the real world and fails to be inclusive then Google will be swept off their feet by the competition and as long as that is the case then I don't see what I need to be so cautious about.

And problems such as political bias run far deeper than some recommendation system. I feel as though these things are just trendy to talk about.

AI should be immune to https://en.wikipedia.org/wiki/List_of_cognitive_biases and it's not possible;

IMHO, a scientist capable of striking any tone in their work, positive or negative, is doing marketing / PR work rather than science.

There's nothing wrong with that. Just like any other member of society, a scientist may want to contribute to a (for-profit or non-profit) organization of their choice, be it for material or spiritual rewards. However, I think it's important not to get science and marketing confused.

So, I think a more precise title would be "Google told its scientists to strike a positive tone when they use academic papers as a marketing tool".

Any scientist writing prose will necessarily strike a tone. Unless they’re just publishing raw data, the way you use that data in your prose will lead to a different tone. I have read a good deal of historical and modern scientific papers and they all had different tones.

When I was young and (more) idealistic, my department chair pulled me aside after I’d been dogmatic about something and said “everything is political... The sooner you realize that, the better off you’ll be.”

It took 10 years for that to really sink in, but I knew that every paper has an angle well before that. There’s a position of inevitability about AI around Google, and, to an extent, I agree. Unlike tobacco, alcohol, firearms, cars, fuel, AI is applied mathematics. It’s virtually impossible to restrict functionally, requiring macro behavioral constraints on businesses and governments that are very difficult to enforce if you want to regulate it. Furthermore, ceding the technological lead and removing the business incentive likely has defense (read: offense) implications on the global stage.

Some things are going to be hard to explain. Some skin tones, hair styles, accents, pitches, timbres, writing styles, etc. may have more/less separable entropy, and that’s going to inevitably make models seem racist/sexist for some applications. However, we’re pretty far from those edges. Instead, our model biases have likely been driven by our datasets, drawn from our coworkers, family, and friends. In this industry, that’s currently a recipe for over sampling in some demographics and under sampling in others.

The thing is - I don't think the bias in AI is fixable. The way we fix it is to regulate (not self-regulate, Google, Apple, Facebook and other AI companies aren't trustworthy to do it right) where AI is allowed to be used and how much of a negative impact of a persons livelihood can it impact.

In other words, we need to use all the power we have to avoid "The computer said NO, you're homeless/jailed/poor forever now" scenarios.

Unfortunately a legal solution is weaker because it has to enumerate the precise conditions that are unacceptable. Do you ban the logistic function? Tanh? Statistics?

Actuarial tables have been a part of decision-making for a very long time and it's not possible to form a comprehensive legal statement of "all decisions must be made in good faith with respect to inherent biases and to the most rigorous epistemological standards".

Funny enough, Aggregability is NP-Hard, so comprehensive consideration of the larger set is not actually practical/possible.

Here's the proof: http://www.cs.utep.edu/vladik/2006/tr06-29.pdf

Perhaps there are mechanisms around performance under external test, auditing, or safe harbor due to documented good-faith, but it is not possible to guarantee correctness for the full set from a subset.

Similarly, some approaches will plainly fail to perform well with subgroups of people due to sensing technologies. That may appear as model bias, but it's really a matter of physics and available entropy.

For the time being, mechanisms that use machine inference as an aid rather than a guide (leaving the judgment to the human in the loop) are probably the best we can hope for.

>Some things are going to be hard to explain. Some skin tones, hair styles, accents, pitches, timbres, writing styles, etc. may have more/less separable entropy, and that’s going to inevitably make models seem racist/sexist for some applications.

We all know that if you have garbage as an input the output can be anything. So a professional data scientist should do his job and analyze the data first , identify the problems and if it can't be clean go in the world and collect better data.

If you are a high paid professional you should understand that say for face recognition you should get a correct sample size, correct distributed (sorry I am not a professional so i don't know how they should do their job but I know that there are statistical tool that will measure things like this) , instead it seems developers use as input for their experiments some garbage data that they can get their hands on.

If all you are wanting to know is "will this work" it is way to onerous to focus on getting a perfect dataset. If it gets into a product though, that is really pretty bad, but it may be a consequence of their development lifecycle or any other org issue rather than a failure of this or that data scientist.

>If all you are wanting to know is "will this work" it is way to onerous to focus on getting a perfect dataset

If the definition of work is relaxed and equivalent to "is better then random and our marketing can sell it"

If you are a professional and you data implies that race predicts school grades and you say "it works" instead of "something is not right, let me check the financial status or house size/family size" then you are an impostor, you plug libraries with code from Stack Overflow until it works, doing the right job would give you better results and fix the bias. If you work for Google,Facebook or other giants I am sure there is not an excuse that we don't have the money to hire competent people and cleanup the data or gather correct data.

Think about it, because some person/company is incompetent you make life harder for millions of people, why we try to find excuses for not doing the correct thing, we need to demand that this AIs with big consequences are created using mathematically correct methods and when someone reports potential problems we need to stop excusing the guys that want to silence them because bad PR.

Pick randomly a list of top papers in ML, for example https://www.kdnuggets.com/2020/01/top-10-ai-ml-articles-to-k.... Most (if not all) all of them have no "positive" or "negative" tone, as far as I can tell.

> every paper has an angle

Not really sure if this is true in general. May be true of commercial entities, but I really hope this isn't true in general.

> Furthermore, ceding the technological lead and removing the business incentive likely has defense (read: offense) implications on the global stage.

Does it though? Uncovering faults and cracks of a system does exactly the opposite. Discouraging experts from looking for those faults may give the perception of lead, for a while, but is a dangerous mistake to make in the long run.

Interfering with experts by pulling them out of, say, academia by paying them more and turning them into your PR tools, I think, is detrimental to the advancement of technology. We may be losing a whole generation of scientists to AdTech when they could have been doing more worthwhile work elsewhere(which, by the way, is why we are here in the first place - someone toiled away at a university to come up with a genius idea to design an artificial perceptron).

As to your claim about whether these companies will play a pivotal role in paradigm shifting technology advancements like fundamental research has in the past remains yet to be seen.

I don't think that uncovering faults and cracks is a disincentive. Prohibiting use of machine learning techniques and technologies definitely reduces the addressible market, and, thus, profit incentive.

There are plenty of reasons to do research and make strides, but "big pot of gold at the end of the rainbow" is one we have to acknowledge the existence of.

As far as paper angle goes, sometimes the angle is "what we spent the last year doing is noteworthy enough to warrant publication".

Read every paper skeptically. Make it prove to you that it has the data, rigor, and reproducibility to support its claims.

This is such a tired argument.

Sure, politics is in everything down to the design of the armchair you are arguing from. That however doesn't fucking mean that all papers shall be considered equal?

Do you think you would be able to differentiate the value of a census statistics from a 2015 paper written by the state of Nevada from a 1930 census statistic paper from Nazi Germany? Excellent!

Because very obviously there is a line where science crosses into bullshit. Sure, the line is multidimensional, ever changing and very blurry. As you say, from finance to family everything factors in and there is no true objective way of doing science. That does not however mean we as humanity can get away with defeationist pseudo-enlightenment by just giving up every analysis of nuance or motive.

What Google is doing here is not in the territory of science anymore - you can and should argue that - and should be treated as such. The price of freedom is eternal vigilance, whataboutism won't cut it.

Edit: grammar

> 1930 census statistic paper from Nazi Germany

Minor point, but there was no Nazi Germany in 1930. They gained control of the government in 1932-1933.

I’m saying that papers should be read with a default position of skepticism and consideration of the motivations of the authors (and they all have an angle). When I read an ML paper with authors from Google, I assume they have an angle. When I read an ML Ethics paper with authors from Google, that already went double. Now it’s triple.

So look at my argument differently. Every single scientific paper you read should be read as non-objective as a starting position. If it can overcome starting from that position with data, methodology, and reasoning, it’s a solid scientific paper.

When scientists write results sections and abstracts they strike a tone. They make an argument for why the raw data is meaningful. Scientists interpret data, which involves a layer of human analysis that is not objective but is attempting to find the correct language and narrative to describe the results of an objective, quantitative process. Outside of the methods and data sections, academic science writing absolutely has to strike a tone. And especially when you are seeking grants and career progression one of the tones you have to strike is to highlight the importance of the work.

Once in a science course in college, I had to present a paper to my class. I tried to present it "objectively" as you would describe it, avoid striking a tone or making judgements. My professor told my presentation was technically correct but failed to highlight the broad significance of the research, which comes from tone. Science writing requires selecting a tone of voice.

'is doing marketing / PR work rather than science. There's nothing wrong with that'

I like how we learned nothing from decades of lies and billions of damages in big tabacco, asbestos, lead in petrol and climate change.

Whats next, can we tell aircraft safety inspectors to strike a positive tone and declare a damaged airframe flight worthy? How about rating some securities in a positive tone?

Lets call it what it is, corruption

Scientific papers often have a "research implications" or "implications" section that generally is optimistic or pessimistic depending on the results. There's nothing wrong with this, so long as the implications are not steering the results.

$COMPANY is a private company and they can censor however they see fit. If you do not like it, you can start your own company and do it your own way.

If you agree with this statement for one company you should agree with it for another.

I don't agree with it for any company. When the private sector crowds out basic research and takes over the function of public institutions, there are three reasonable scenarios. 1. they let the researchers do what they want and science goes on, 2. they're regulated so they can't compromise science, 3. they do what Google does, in which case it's time to take the gloves off and go to 2. because the integrity of scientific research for its own sake is more important than whatever corporate dystopia Google is trying to build.

This take does not make any sense to me. The scientific community is responsible for peer reviews and journals can operate however they’d like. Google is responsible for google and can submit/censor/only pay for research that they want to pay for. Good luck finding any academic that is not beholden to their source of funding.

Academia is not the “uncompromised land of science” that it may seem to be. What google is doing is par for the course. I have about 50 other complaints about academia before I get to my complaints about private companies censoring research (which I believe is a very harsh take on the situation at hand.)

In Terminator Genesis, the AI was put in a good light also- on billboards and such.

It didn’t work in that movie, though. The AI still ended up being hostile even when the public embraced it.

Why are we xenophobic about AI self-awareness? idk

Reply on news.ycombinator.com