Hacker News Reader

Comment by llm_trw | Hacker News Reader

Because I don't want to spend two years fighting the second referee when she's fundamentally misunderstood the point of my paper.

I'm no longer in academia. Either take what I put up on arxiv or leave it. I _really_ don't care.

View on HN

You actually touch on an interesting point. Is peer review still necessary? When there is limited journal space, sure. But now that we have effectively unlimited space, let it be reviewed when people go to cite it. We've already seen a lot of peer reviewed papers be retracted later, so should we just accept the reality?

Yes. The crap people publish even with peer-reviews is bad enough. A few subfields in CS have bitten a couple times by relying on non peer reviewed ecosystems.

Peer review is the backbone of journals, and (some) journals serve an absolutely critical function in academia: they increase the signal:noise ratio. Most of what is published is noise; without the curation provided by (certain) journals, anything of significance is very likely to be drowned out.

As a casual example in the biomedical sciences, the Journal of Biological Chemistry has an output of ~30,000 pages per year, most of which is 'noise'. That's just ONE journal. The journal Cell, on the other hand, has an order of magnitude less, most of which is 'signal'.

EDIT: This is not to say the peer review approach doesn't need work, and lots of it. The whole current approach to research needs an overhaul. I'm just saying it's a bit hasty to throw the baby out with the bath water.

Peer review is an important signal to potential citers. If everyone has to fully read and understand every paper in order to (responsibly) cite it, there's gonna be a lot less research. Given the exposure of how much bad research there is, maybe there does need to be a slowing and focus on quality, but I think we still need peer review, although it definitely need to be reformed somehow because it's clearly broken.

We need to take a serious look at the incentive structure in academia because it's not guaranteeing the scientific results that we expected it to. I don't think we should just abandon the system though.

Peer review can be very simply replaced by skimming the abstract. If the abstract doesn't make sense the paper doesn't make sense either.

All the disciplines which use arxiv as their main journal are doing well enough without peer review.

No, it is not necessary. Just as it was not necessary for the first 300 years of the Royal society’s existence. “Peer review” used to be just a stand in phrase for the marketplace of ideas—seeing how your peers evaluate your work in a public process of getting published and sending letters to the editor. Only in living memory has it been warped into this pre-publication gatekeeping process.

Necessary to share ideas? No.

Necessary to share important ideas? Maybe.

One of the aspects of journals that I don't see talked about much here is the curation of articles. Saying (good) journals aren't necessary is like saying journalists aren't necessary to get your news in the internet era. Not all information is necessarily good, and we often rely on systems/people to help winnow the amount we have to sift through. This is important in a society where the sheer amount of information developed far outpaces what we can consume in several lifetimes.

It's more like saying that maybe we don't need the NYT, the people who lied about Iraq: https://www.theguardian.com/world/2004/may/27/media.iraq, to get good quality journalism when the internet exists.

Journals are in the same boat. Any good they have done is dwarfed by the few 'honest mistakes', like Alzheimer's cabal: https://www.statnews.com/2019/06/25/alzheimers-cabal-thwarte..., which have done more to stifle science than anyone since Stalin picking which biological theories are socialist enough to be true.

I think you're conflating the notion of perfect curation. I don't think anyone is claiming that journals or journalists are infallible. They certainly make m, although I don't think I agree that the good is "dwarfed" by them, except when used as a direct attempt to undermine credibility by adversaries.

Put differently, do you think bad ideas spread more easily with the internet? The current research seems to think so, and if you agree, I don't see how that mitigates the spread of bad information. If anything, it exacerbates it.

I think that every bad idea spread by the internet dwarfs the harms done by ideas spread by government. Every issue with the middle east over the last 20 years is the direct result of a crusade launched on a lie enthusiastically repeated by the same people claiming the moral high ground today.

While not always true my metric for clear/understandable is for other people to understand it. This usually supports my argument when people show me a document and I have no idea what its saying, they argue its perfectly clear... my definition was for people other than the author to grasp the intended meaning.

Peer review goes beyond simple issues about clarity or misunderstanding. In particular, peer review is sometimes seen as an adversarial process.

Often, the reviewer will not understand because he is not the intended audience. Other times, he will understand but he just doesn't like your method, because he is working in an opposite direction. Or maybe your method is a direct competitor of his and yours work better, which incentivizes some people to block your work.

Or your paper goes against the current paradigm or is otherwise politically unpalatable.

> Other times, he will understand but he just doesn't like your method, because he is working in an opposite direction. Or maybe your method is a direct competitor of his and yours work better, which incentivizes some people to block your work.

Oh you mean those phantom "off topic"/"out of scope" reviews.

If the intended audience is reading it, I would agree. However, many reviewers seem to be assigned and agree to review a topic that they are ill-equipped to understand without a substantial amount of background knowledge.

A good reviewer in this situation will review the referenced papers to bolster their understanding. A bad reviewer will expect everything to be spelled out within the manuscript, and, unfortunately, the length limits often don't permit that kind of write-up.

Precisely. The continued hyperspecialization of science shrinks the potential pool of qualified reviewers to zero. I rarely get knowledgeable reviews these days. Instead of disputing my methods, they complain about the font or figure styles, or raise questions that weren't answered in the paper because they are common knowledge in the field.

Reply on news.ycombinator.com