Should scientific papers be anonymous?
statnews.com
statnews.com
In the real world, having names on papers is a really good thing. In real academia, I've heard professors say "well that work was done by so-and-so, and they're bad researchers so I don't trust their results".
There's this popular misconception that just because something is published in a peer-reviewed journal that it must be true. Nobody in research actually believes that. All it means is the authors were able to convince the reviewers and the peer reviewers to publish it.
Just as important would be institutional anonymity. A paper from Nowheresville should be considered equally to a paper from Harvard.
Finally I think papers should be required in their headings to disclose sources of funding for the work therein, although this would have to be done later so as not to indirectly compromise the anonymity period.
Science is the organic collection of all people and resources and knowledge engaged in figuring out how our world works. Having blind papers would mean I would just have to spend an extra 10 minutes reading each paper to figure out what group wrote it.
The fact that Prof. Ivy League gets her results recognized more easily then Prof. Nowhereville is, actually, an ok system. Because Prof. Ivy is more likely to be pursuing interesting problems to the rest of the scientific community, she's more likely to be trusted by other scientists to be a good scientist (i.e. not careless), and the system supports people who have "made it" through the academic gauntlet.
If Prof. Nowhereville actually finds something interesting, pursues publication, passes peer review, and it gets published then great. His results are probably worth the attention of the scientific community. But the fact that the barriers are higher is ok, because he hasn't proven himself to be as trustworthy.
I think this is ultimately a manifestation of "scientific deontology ethics", a fantasy that we should approach each paper with identical, non-prejudicial priors and evaluate it on its own merits. (It is the scientific analog to the reductio ad absurdum of maximum non-prejudice in human affairs, where one never considers any background about a person before dealing with them.) This position only seem tenable to those who haven't yet had to wade through the morass of terrible and misleading result in the literature.
In other words, this sort of idea is not a consequentialist attempt to solve a problem, it's a deotological attempt to stop unfairness.
The community would be forced to develop tools and documentation to judge new papers based on actual content and aggregated opinions, which seems like a better way to filter poor-quality human-written content (we're not talking about email spam here). Plus, by making the process explicit and visible, it'd be easier for an outsider to engage with new scientific literature too.
Perhaps that's an approach we could use in software development. It's mostly how I use HN or Reddit -- I read plenty of comments, and only ever look at the name if the comment provokes interest in the writer's background.
I think you've got it backwards. First make the good tools, then stop using the bad ones. You don't force people to develop good Bayesian spam filters by banning whitelists. Rather, the Bayesian spam filters get invented, then people are happy to stop using whitelists.
Likewise, scientists would love nothing more than to have the tools that enable a transparent frictionless utopia free to barriers to entry. They'd get access to a few new ideas caught in the spam filter! But we don't get there by anonymizing papers. That would just lead (almost instantly!) to good scientists whispering in the ears of their colleagues about which papers they just published, or to take a look at that new paper by so-and-so, etc. (And then the only people who could access the metadata would be those who are already at good institutions!)
Firing people to either develop tools or have a terrible situation on their hands doesn't necessarily mean they would develop those tools.
If you push someone into water, it forces them to either swim or drown, but it doesn't necessarily imply that they swim.
Both have pros and cons. One con with having names on a paper is that it can give it attention purely because of the authors name at the expense of something else.
However I agree that scientist wouldn't do it (which kind of supports the idea of nameless papers IMO)
- GitHub, FigShare, institutional repositories etc. for code and data
- Information on the location of the study
- Statistical packages used (hope you didn't author the one you're using)
Of course, anonymity will be shattered the moment anyone talks about their work at a conference. Furthermore, several systematic reviews and meta-analysis studies I've worked on benefitted greatly from being able to email the authors directly with questions.
I'd also be concerned that people will mentally assign things to a "Big Name" in the field anyway. "Ah, another paper on HAART. Must be out of so-and-so's lab..."
This was more a note that, when the paper is accepted, those identifying things go back in. The proposal to have papers be anonymous would have to leave them back out, stripping the paper of important context, and making open science a bigger pain than it already is. Which is kinda contrary to the goal.
In practice, someone who really wants to break the double-blind veil probably can. But I don't think that's a problem, as very few people are consciously biased, and will actively seek out such identifying information. By removing names and affiliations, and pretending all of our work was done by someone else, I think that's enough to help people avoid unconscious bias.
"(AUTHOR et al.) performed a small scale lab study. (ANONYMOUS SUBMISSION) performed that exact same study, on hundreds of people, and expand on the work of authors..."
Makes me wonder if you could implement the exporting of a "review" version in LaTeX, similar to a debug mode -- set a flag, and any content with an {ID} wrapper around it gets left out or is substituted for something anonymous.
While you can say "Remove obvious links", to many experts $Topic in $Location is enough to allow a pretty good guess of whose lab its out of. Or the name of a long running study. And I'm not going to maintain a whole separate Github account to anonymize code distribution.
Unfortunately, I saw submissions get down voted not because of the content, but because of some long standing intellectual feud (I found this out after I became a sought-after "tie breaker." The journal editors figured out I was indeed casting votes based solely on the submission in hand).
Journals want high-quality submissions from well-known authors in order to maintain the brand. They also want ongoing subscription fees.
Authors want to be published in good journals, but they also want raw publication count to be high. So the primary incentive may not be intellectual honesty.
Reviewers often are in competition with the authors of the papers they're reviewing, hence the term peer-review. They have an incentive to prevent publication of competing work.
Readers want all knowledge to be free, so they have an incentive to not pay for journals. They also want to know which works are important and which are crock, so name-recognition comes into play.
I've probably left something out, but you get the idea. Coming up with new ideas is fun, but until you align the incentives, nothing meaningful will happen. And you've got hundreds of years of inertia and thousands of careers in the balance.
This seems backwards to me. There isn't a fixed amount of interest in a field that is captured by peers. If others in your industry are getting attention then it draws attention to your industry as a whole. I'd think reviewers would be biased towards having competing work published not against.
Plus, everyone can tell who did what-- if not exactly who, then one of a small (5-10) group of people who are likely in the same geographical location.
I understand though that simple anonymous comments don't cut it. It can devolve into youtube comments quickly -- personal attacks to the authors, low effort comments, spamming, "trolling", etc.
Those sites need a private reputation system: you can vote comments but the commentator is anonymous; it's hidden reputation should help sorting the comments.
Have you looked at PubPeer? There have been some very interesting posts there.
We should remember that there are three major forms of incentive in the scientific community :
1. Prestige (requires name) 2. Salary and grants (requires credentials, and hence names) 3. The joy of analysis and discovery (doesn't require names)
(I might have missed some forms of incentives, but these seem to be the major ones to me.)
Anonymity will take the top two incentives away. The second one is a very practical one. There's an existing competition for salary and grants, a pretty brutal one in some areas where money is scarce. We need to solve this problem first before the concept of anonymity can be realistically entertained.
I have heard other proposals for reducing bias that seem more tangible. One of them is democratization (partially or fully) of peer review in a centralized publication system (like arxiv).
Peer review can apply the scientific method to the paper well enough. This may take time. That is just fine.
People having their names on papers helps with motivation to publish in most cases, and it's good to understand who contributed what when trying to understand how an idea has formed and who might be with collaborating or talking to.
Some topics may be difficult politically. Those are special cases that can and should warrant doing whatever it takes to insure publication and the safety of the scientist.
There are also ethics. Anon publishing may well increase questionable research. This should fall into the special case bucket too, as it overlaps with politics.
Which makes me think maybe HN should have some mechanism to say "This article is related to this other one", and point to the HN submission for it.
I guess our user comments here in this subthread are that mechanism. By the way, this whole thread is an implementation of the new HN algorithm that revives threads that are missed on first submission and gain upvotes later. I notice that is implemented by resetting the submission date on the submission. (I was the original submitter, so I remember when I first submitted this thread.)
Imagine a situation when some climatologist makes a discovery that global warming is a fluke. In the current political climate no reputable journal will accept his article on this topic. Moreover, no scientist will even SUBMIT one because of the possible damages to his reputation.
On the condition of anonymity, it may stand a chance.
By mandating blind reviews, the bias can be removed easily.
There are many who give good/bad reviews just by looking at the authors names, not everyone of course.
I am not sure why blind reviews are not mandatory by now
This wouldn't work in almost any field because the community can already tell with high accuracy who the peer reviewers are, and it's usually easy to tell who wrote a paper by the language style and the conclusions.