Researchers retract controversial female mentorship paper after Twitter outrage

Virtually all research can be subjected to valid criticisms and scrutiny. However, these days which research receives that scrutiny (along with social media outrage) is mostly a function of what the conclusions of the research are and how those conclusions align with political ideology.

A simple thought exercise here is to think about how a research paper with the opposite conclusion would have been treated. It would have been applauded, shared broadly in social media, and would have gone unchallenged rather than retracted. In short, science has been corrupted by political power because we've let one side exercise the power of censorship and deplatforming.

I'll be honest, my political reflex muscle is too tired to pull anymore. Let's take a look at the study, because my mental reserve of scientific attention is still O.K. (under-tapped, if anything.)

Here's the retraction letter written by the authors themselves: https://www.nature.com/articles/s41467-020-20617-y

They write,

> The three independent experts commented on the validity of the approaches and the soundness of the interpretation in the Article. They supported previous criticisms in relation to the use of co-authorship as a measure of mentorship. Thus, any conclusions that might be drawn on the basis of co-authorship may not be automatically extended to mentorship. The experts also noted that the operationalisation of mentorship quality was not validated in the paper.

I think they're saying that co-authorship does not imply mentorship. That sounds reasonable on its face to me, and I wonder why they didn't notice that disconnection before publishing. I had to look up the word "Operationalization," apparently it is a social science term for the way in which researchers choose to measure something. (An example of operationalization would be, "let us use citation count as an operationalization of career success.")

Now, let's take a look at the original paper to decide if the coauthorship=mentorship thing was really as straightforward as the notice makes it sound.

https://www.nature.com/articles/s41467-020-19723-8

>For any given scientist, we consider the first 7 years of their career to be their junior years, and the ones after that to be their senior years. Whenever a junior scientist publishes a paper with a senior scientist, we consider the former to be a protégé, and the latter to be a mentor, as long as they coauthored at least one paper with 20 or less co-authors and share the same discipline and US-based affiliation; see Supplementary Note 3 for more details.

It does look like their measure was a little more sophisticated than coauthorship alone, but it also appears that they never checked to see whether their measure worked. Nowhere in the study did they, say, ask a bunch of coauthors if they really were in a mentor-mentee relationship. Curiously, they did survey pairs to ask other questions.

Here is where it gets more interesting. From their paper, they write:

>To verify whether this is the case, we sampled 2000 scientists whom we identified as protégés, to ask them about their relationship with their mentors. [...] Out of those 2000 scientists, 167 completed the survey; see Supplementary Note 4 for more details.

The many non-respondents could easily be a set that contains their mis-identified pairs. A scientist who is not actually in a mentor-mentee relationship would be unlikely to respond to a survey asking them about their relationship. The number of non-respondents is more than ten times the number of respondents.

It's conceivable that coauthorship is correlated with mentorship, but the authors skipped a big step in their chain of deduction.

"The study, published on 17 November by researchers from New York University, Abu Dhabi, combed through more than 200 million scientific papers to identify several million mentor-mentee pairs, then tracked their co-authorships and citation records to evaluate the impact of mentorship. Their conclusions, including a finding that “current diversity policies promoting female-female mentorships, as well-intended as they may be, could hinder the careers of women,” angered many researchers. Critics attacked both the study’s conclusions and the methods used to reach them.

In a retraction notice published today, the authors wrote that they recognized the validity of some of the complaints, including concerns about “the use of co-authorship as a measure of mentorship.” "

Wow, they used co-authorship == mentorship and from that assumption concluded that women-women mentor-mentee relationships result in worse outcomes? That's one hell of a reach. How did this make it past peer review?

But, was the paper factual?

These kinds of mob-mandated retractions, especially in the wake of the ongoing replication crisis, only bolster conservatives' skepticism of science and their suspicion that it has been corrupted for political purposes.

I'm seeing more and more outright ridicule of science from conservatives now, or at least ridicule of the emphatic, unqualified appeals to it: e.g., "the science is settled!" "trust the science!"

Is censoring papers that support wrongthink really worth half the country losing in faith in science altogether?