The peer review industry: implausible and outrageous | Hacker News Reader

The peer review industry: implausible and outrageous

the-tls.co.uk

133 pointsjseliger49 comments

Open access, as is mentioned in the article, is both important and difficult to achieve. As a PhD student or associate professor, you work on a strict schedule to publish or perish. The only acceptable publication venues are the 4-6 respectable journals in your field. You have no time or power to push for open access.

Only large research institutions like (I’m familiar only with the CS research community) MSR, Google Research, Facebook Research, ... as well as already tenured professors can really force the issue.

I work at a smaller industrial research lab. We try to push for open access whenever we publish, but publishing in non open access journals is important to maintaining our perceived prestige, which in turn is important in hiring (especially when we poach from academia).

I publish pretty regularly and try to be selective, but ultimately the push has to come from an institutional level such as universities or large industrial research labs to say “we will only publish in open access journals”. This is especially true in the US where EU style legislation is a long way off.

do you put your work in the arxiv?

Not the OP, but I've started putting all my first-author papers in the EarthArXiv [1]. The original ArXiv only accepts preprints from a few fields, but the Open Science Framework has recently made a federation of preprint servers that fills many of the gaps, and can be searched in one place [2].

[1] https://eartharxiv.org/ [2] https://osf.io/preprints/

Large industrial research labs have the same problem. They need to maintain their prestige for recruiting purposes.

At the same time, this seems to select the worst of the candidates: those who care about prestige.

> Only large research institutions like (I’m familiar only with the CS research community) MSR, Google Research, Facebook Research, ... as well as already tenured professors can really force the issue.

Well, they can to a certain extent, by submitting their work to open access (preferably fair OA [1]) journals, thus increasing their prestige. I think a more important role here lies with funders, though, and more specifically the people that decide who to fund (although these often are researchers as well).

By currently strongly relying on journal brand names as indicators of "excellence", they're providing the incentive that keeps/forces academics to publish in traditional, paywalled or high-APC journals. They need to find other ways to decide whom to give funding to. (And perhaps have less of a focus on an ill-defined idea of "excellence" as well.)

Disclaimer: I'm part of a just-launched initiative that aims to provide such an alternative. I'd love to hear people's thoughts about this though.

[1] https://www.fairoa.org/

What to do about the entrenched scientific publishing-funding complex has been a concern since the first full text of journal articles started appearing in the 1990s.

The response to date has been for funding agencies (nonprofits and governments) to require publication in open access journals as a condition of funding research. This has happened in both the US and EU.

However, I'm pessimistic this approach will address the problem, and it could well create a number of unintended consequences. For one, publishers that have introduced OA models can charge thousands of dollars to the research teams publishing the work. That could mean fewer publications, and more information silos.

The article hints at, but doesn't fully describe another, more pressing problem. Peer review is in trouble. The number of papers the average researcher is being asked to review is increasing while the abundance of papers means the few bother to read them.

I suspect the solution lies somewhere else entirely. Post-publication peer review is one option, and Fighare has built an interesting model that could be a component of a future system:

https://figshare.com

Regardless, the problem is cultural, not technical. The crazy economic incentives described in the article work because everyone involved has a (sometimes perverse) incentive to keep it in place. Until that changes, it will be mostly status quo ante.

> the number of papers the average researcher is being asked to review is increasing

The quality of review becomes proportional to the impact factor of the journal. And given that the high impact journals have not substantially expanded the number of articles they publish, it might mean that there are now more candidate reviewers.

What has become substantially more burdensome is the number of funding applications that (esp. young) investigators have to submit per year. http://www.pnas.org/content/112/2/313

> Post-publication peer review is one option, and Fighare

There are interesting proposals. E.g. elifesciences proposes to publish everything, under the condition that the reviews will be published eponymously along with the paper

no proposals, the review feedback and author response to the review are published. Here is an example: https://elifesciences.org/articles/35082#SA2

Peer review at elife works slightly differently. The author(s) are not subjected to multiple rounds with multiple reviewers, instead the review feedback from the peer reviewers is aggregated by the editorial staff before being sent back to the author. There may be multiple rounds of this however.

I think the OA requirement is a good step but, as you mention, not enough to get us the entire way there.

The EU and several European funding bodies have made the next step [1], which is to require funded researchers to only publish in journals in which all research is Open Access, pressuring journals to flip their models, and by putting a cap on the publishing fees those journals are allowed to ask.

That said, that still doesn't get us all the way there. Firstly, it remains to be seen how publishers will deal with that - for all we know, they'll set up sibling journals for European researchers. More importantly, though, is that they haven't specified yet how they will decide who gets funded, if the name of the journals one has published can no longer be a factor.

Then again, I'm hopeful that this drastic plan will shake things up enough for a solution to arise eventually.

Disclaimer: I'm part of a just-launched project that aims to provide a solution for this.

[1] https://www.scienceeurope.org/wp-content/uploads/2018/09/Pla...

> The number of papers the average researcher is being asked to review is increasing.

"No" isn't just a word, it's a full sentence. Just refuse to review for free, demand an appropriate consultation fee instead. If the journal actually pays you---good. If it doesn't, you did your part to make their exploitative business model unattractive.

I just realized that this analogy of the academic publishing industry is also quite similar to that of the search engine industry.

Most web pages are provided to the public free of charge, but they typically don't obtain them directly (technically they can, but it's often too tedious). A search engine takes them and redistribute to the public. They do it by using the metrics mostly provided by the (unpaid) public: PageRank. And they make a hefty profit for doing that. They also amass a lot of power. Both industries are essentially doing the same thing: content curation.

The difference is that search engines don't get complaints like academic publishers.

I think the reason is that the academic publishing is much simpler and actually more transparent than search engines; you can easily see the inner workings and the flow of money. On the other hand, people don't really understand how Google makes so much money. We're effectively blinded by the complexity.

The search engines have sidestepped grief because of their third-party pays model - advertisers fund their content curation and discovery expenses and the system retains open access at no (direct, monetary) cost to the consumers.

The peer review process might be more expensive/intensive than PageRank for all I know, so perhaps journal publishers are doing much more work than search engines.

Perhaps if journal publishers used an ad-supported model they wouldn't be demonized so readily since they'd be incentivized for open access since they're selling eyeballs

I'm not advocating, just working with the parent's comparison.

> The peer review process might be more expensive/intensive than PageRank for all I know, so perhaps journal publishers are doing much more work than search engines.

The peer review process is labor intensive, but to add insult to injury, publishers aren't the ones doing it and they don't pay reviewers. It's all done for free by researchers.

At best publishers add some proof reading services, but in many cases not even that.

> (4 paragraph analogy of implausible and outrageous scheme of bakeries which is total fiction). > ...Yet the story is broadly analogous to what happens in the publication of academic journals.

While I think there are lots of problems with the academic journal system, please, please, please do not reason from analogy like this article does. This is a complex topic that deserves being reasoned about directly. Presenting an analogy which is obviously artificial just primes readers with a false, oversimplified narrative. Don't do this. This is highly misleading, and the bias that results from the priming is corrosive to clear reasoning.

What I don't understand is why peer reviewers agree to perform uncompensated labor on behalf of these gargantuan, highly profitable publishers. What is the benefit to them, other than the warm glow of fulfilling one's professional obligations?

Often it is a favor to a colleague that is serving as the (also unpaid) editor to get a line on their CV. Given that the fields are highly specialized, you tend to work with the same small community of people (writing articles and getting project funding together, reviewing each other's articles, etc.), so it pays to be well-liked among your peers.

Its very hard to turn down a request to review a paper you are qualified to comment on from an editor you know, or who regularly handles your papers. Editor X from Journal Y sends you a paper which you are well-qualified to review. You've submitted 5 papers to Journal Y in the last few years, some of them handled by Editor X. Are you really going to turn down Editor X's request?

What if the paper is written by Author Z, whose work you are familiar with and whose papers you read anyway. Why not review it?

And so on.

Mostly it's the latter, out of a sense of duty.

But there are other motives. Regular reviews give you some perspective that you otherwise wouldn't get. For example, you see what mistakes others make and that can help you improve your own papers. Another motivation for me is sometimes that I don't want the paper to be reviewed by someone less qualified in my areas of specialization.

Ever noticed that the person who appeals to your sense of duty is always your boss who doesn't want to give you a raise?

Aside from the other explanations, you also get a slight competitive benefit. If the paper is good, you get to use the ideas in the paper ahead of other people. Though everyone uses arxiv now, so not so much anymore.

Also you get a sense of power anonymously judging your colleague's work..

Reply on news.ycombinator.com