Hacker News Reader

Comment by tooltalk | Hacker News Reader

Please go read the link first. Jeff clearly states that Google has a review protocol for journal submission which requires a two weeks internal review period.

Timnit shared the paper a day before the publication deadline, ie, no time for internal review, and someone with a fat finger apparently approved it for submission without the required review.

View on HN

That's not under dispute. What's under dispute is:

1) Is the review protocol that requires a two-week review period a peer review process intended to maintain scientific rigor, or an internal controls process intended to prevent unwanted disclosure of trade secrets, PII, etc.?

Repeating the comment at the very top of the thread:

> Maybe different teams are different, but on my previous team within Google AI, we thought the goal of google's pubapproval process was to ensure that internal company IP (eg. details about datasets, details about google compute infra) does not leak to the public, and maybe to shield Google from liability. Nothing more.

If it's not a scientific peer review process, arguments about why scientific peer review is generally anonymous are irrelevant, just like arguments about why, say, code review is generally not anonymous would also be irrelevant. It's a different kind of review process from both of those.

2) In practice, is the two-week review period actually expected / enforced? Other Googlers, including people in her organization, are saying that the two week requirement is a guideline, not a hard rule, and submissions on short notice are regularly accepted without complaint:

https://twitter.com/le_roux_nicolas/status/13346245318860718...

https://twitter.com/ItsNeuronal/status/1334636596113510400

https://twitter.com/lizthegrey/status/1334659334689570817

(I don't work for Google, but I work for another very IP-leak-sensitive employer that does ML stuff, and we have a two-week review period on publications. The two-week rule exists for the purpose of not causing last-minute work for people, but if you miss it, it's totally permissible to bug folks to get it approved, and if they do, it's not considered "someone with a fat finger." It certainly doesn't exist for the purpose of peer review - it's assumed that the venue you're submitting to will do review, and I think everyone understands that someone from your own employer isn't going to be a fair peer reviewer anyway. There is a "technical reviewer" of your choice, but basically they just make sure you're not embarrassing yourself and the company, and there's no requirement for how deeply they review. I think I've gone through the process twice and missed the deadlines both times.)

So, if this "rule" exists on paper, but only exists in practice for her, then this is the textbook definition of unfairness.

BTW, a couple more examples of Googlers saying this isn't a firm deadline by any means and one-day reviews are quite permissible:

https://twitter.com/william_fitz/status/1335004771573354496

https://twitter.com/mena_gonzalo/status/1335066989191106561 (an intern!)

Papers differ. A short, straightforward, low-impact paper on a non-controversial topic could probably be reviewed in a glance or even rubberstamped. A long, complex, high-impact paper on a controversial topic (or worse, a paper with a fundamental conflict of interest) might take a long time and definitely can't be rubberstamped. The paper at question seems to fall under the latter category? It's like skipping a stop sign; 99 times you do it in your neighborhood with no one around and there are no consequences whatsoever, but that one time you do it in downtown with a cop parked right around the corner and you get a ticket.

I think the "skipping a stop sign" analogy doesn't quite work because there was someone around - someone had to approve it, and furthermore, the fact of the late submission and shortened approval is recorded in the review system. If they wanted to tell people "Hey, in the future, don't do that," they could. There'd be more of an argument there if the common case was that, say, people ignored the system and submitted anyway and hoped nobody would notice.

(... Also, comparing this rule to our overpoliced society where everyone commits some sort of crime and the police just choose who they go after kind of reinforces my point about unfairness. Sure, it may have been strategically wrong for her to not do everything by the book, but if so, it's very interesting that the in-house ethicist has to play by all the rules to not get fired and the practitioners can safely skip them.)

Anyway, the culpability for rubber-stamping this paper is on the person who rubber-stamped it, given that short approvals are commonplace. Saying "You should have known that this approval didn't really count, so it's your fault for going through the normal process and not realizing it should have been abnormal" is nonsense. That's literally the job of the reviewer, and if the reviewer can't do that, someone else needs to fulfill that role. At worst, if they told her on day one "Your job is publishing high-impact papers with fundamental conflicts of interest with the company, so everything needs detailed review from X in addition to the usual process," that would be different. But they didn't. Better yet, they could have flagged her in the publication review system as needing extra review. There were lots of options available to Google if they weren't trying to make up rules after the fact to censor a researcher.

And in any case, she gave advance courtesy notice of the planned work: https://twitter.com/timnitgebru/status/1335018694913699840 Someone could have said something then. They didn't.

> Anyway, the culpability for rubber-stamping this paper is on the person who rubber-stamped it, given that short approvals are commonplace. Saying "You should have known that this approval didn't really count, so it's your fault for going through the normal process and not realizing it should have been abnormal" is nonsense. That's literally the job of the reviewer, and if the reviewer can't do that, someone else needs to fulfill that role.

This is key, and I don't see it being mentioned as much in other comments. It was approved.

Reply on news.ycombinator.com