Comment by iandanforth | Hacker News Reader

Comment by iandanforth | Hacker News Reader

Parent Full thread

This. Failure to cite relevant research is a reason to reject a paper from a conference but that's not the job of an internal review. Research independence is fundamental and why professors are given tenure. Sure companies are companies and don't play by those rules, but you can't claim to be making a fair academic criticism if you're not adhering to academic norms.

If the paper was bad it should have been rejected by the conference to which it was submitted. If the paper was critical of things Google does and it didn't like that, that's pure suppression. And, as you say, academic norms provide a period of rebuttal, it's unclear how much was allowed here.

Secondly there is value of having anonymous review in an academic setting, however it sounds like they tried to enforce that anonymous review for the company, which again doesn't serve the same purpose. People should be told exactly why decisions are being made so they can correct their mistakes in the future. If you aren't willing to invite someone into a decision making process, when a large portion of their research is in biases in institutional decision making, then you have got to know you're in for a bad time.

I'm not very familiar with the norms of corporate research. Is it normal for corporations to give their researchers tenure-like freedom to publish anything they want? I assume that top researchers (who have their pick of institutions) will choose organizations where they can be relatively free. But what do you mean by "fair academic criticism"?

If a company publishes only papers that clearly advance its interests, but the research is sound and independently reproducible, does that bias really lessen the perceived value and quality of the research?

At least in theory, something like:

Fair academic criticism: We shouldn't publish this paper because it has flaws X,Y,Z. (usually followed by an opportunity for the researchers to address X,Y,Z).

Unfair and/or non-academic criticisms: We shouldn't publish this paper because: I don't like the author/authors institution. I don't like this approach, approach X is better. I don't like the conclusions.

The last is potentially relevant here. And yes, clearly the quality and value of research is reduced by filtering based on results you want to promote. Which well may be in a companies best interest, but there are sometimes difficult lines to draw - when exactly do you move from "promoting the companies interests" to "propaganda intended to mislead the public"?

That makes sense, but I don't think of corporate approval committees as being designed to render "fair academic criticism". For example, they might say "the core thesis of this paper reveals our valuable trade secrets and therefore you are forbidden from publishing it externally." That's not fair academic criticism, but it could be a reasonable decision for the committee to make.

Right, as I was trying to convey in the last - there are other corporate interests than research value, and I think all would agree there are situations where they reasonably trump the researchers or public's interest. And vice-versa; "This research shows we have been poisoning the public for 40 years" for most people probably really isn't the same as "This research exposes valuable trade secrets". The lines aren't always obvious.

To me, the Jeff Dean's email sounds like this paper failed to cite some research and as a result made Google look less good than Google actually is. This sounds like somewhat minor academic mistake, but at the same time a major PR mistake.

That sounds like something easily rectified, so why push for retracting the submission?

I think the logic is that the paper would need to be retracted, rectified, and then resubmitted. Since the submission deadline was already passed, resubmission would be impossible.

Submissions deadline is typically for evaluation, not camera ready. It's not unusual for textual changes to make their way in after that point, either in response to reviewer or to improve the text in a way that doesn't fundamentally change the results.

>> Failure to cite relevant research is a reason to reject a paper from a conference but that's not the job of an internal review.

The done thing is generally for the reviewers to suggest additional references to improve the paper. If any mention of related work is entirely missing, then that's another matter, but a reviewer thinking that the authors should cite a particular piece of work (e.g. the reviewer's own paper) is not normally something that could lead to outright rejection.

Look, on paper I agree with you, they can't claim 'fair academic criticism' and censure papers without explanation. But in the real world, Google has an outsized contribution to AI. They are also a company, and need to pay for all the expenses. There are practical, day to day concerns that are essential for their existence. If they can't ensure their finances, the research program stops. So you can't really be absolutely impartial, just 99%, from time to time you make an exception to impartiality. It's like light speed, you can't reach it.

Reply on news.ycombinator.com