It's funny people are raising their pitch-forks about the journals. I guess it's not widely known/understood that the journals aren't reviewing anything ever. So when I say reviewers I'm talking about PhDs in academia that do this work as "community service". So if peer review is fundamental to the scientific method then your tax dollars are being wasted but the bulk of the waste is on worthless, lazy, BS spouting academics not journal admins.
> and practically all of academia just went along because there were no other options.
Arxiv has been around for a long time now. Do you really think "the academy" couldn't have come to some consensus on how to allocate review duties for papers that are uploaded/stored/submitted there? The fact that they haven't strongly indicates that lazy ass peer review as orchestrated by the journals is to the benefit of the academy.
Journal (and conference) reviewers are completely anonymous and there's absolutely no appeal process (rebuttal phase is BS). Does that sound like a democratic/fair/auditable/meritocracatic setup to you? Sounds like review by inscrutable tribunal to me. So if the process were open/community driven (ie democratic) then people wouldn't be able to hide from the shitty job they do.
No doubt - past two papers I've submitted I can basically deanonymize my reviewers based on the other papers/projects they insist my work is inferior to.
Correct me if I'm wrong, but isn't peer review usually volunteer work? With the job of the (usually paid) editors to select reviewers with a track record of basic competence.
> Do you really think "the academy" couldn't have come to some consensus on how to allocate review duties for papers that are uploaded/stored/submitted there?
Actually, there are several proposals! One such system is https://openreview.net/about – it's not the first I've seen, but it's the most recent. (I first learned about these things from a fictional proposal in a Pokémon fanwork: it's far from the realm of niche.)
> Journal (and conference) reviewers are completely anonymous
To reduce the incentives for nepotistic anti-scientific behaviour. The system's flawed, sure, but it works how it does for a reason.
You need to ask yourself how that track record is constituted ie what metrics decide basic competence. Hint: it's not anything that resembles an objective measure of ability or diligence or literally anything. Another hint: it's *extremely* common for PIs to pawn review duties off on grad students - I review papers for my advisor all the time and I have been since my first year.
> To reduce the incentives for nepotistic anti-scientific behaviour.
This is the thin veil that everyone hides behind to cover up the greater harm of inscrutability^1; everyone knows what everyone else is working on in any field. At this point in my career (I'm towards the end of my PhD) I could tell you which group wrote the paper (if it's in my area) just by reading the abstract.
And if you're thinking to yourself well what about new entrants into the field? There's no such thing - you do not learn XYZ hyper niche thing without being part of a long lineage of forebears doing XYZ thing. And if it does so somehow happen, like a student is doing something completely different from their advisor and so they're not trying to publish in familiar venues, well guess what they get face looooong odds getting their work accepted (ask me how I know :)
^1 Like come to think of it, recently I've been reading that the conservative judges on SCOTUS think proceedings should be closed. And you could make some sort of similar nepotism case about that, or some other conflict of interest type thing (in favor of closing the proceedings) but we all recognize that fundamental to a free and fair process is that it's open and auditable.
Unless the reviews are submitted in the grad students' names, that sounds like academic misconduct. Can you “anonymously” report this without too much backlash? (I assume the answer is no: you clearly know it's wrong, and haven't done anything about it yet.)
> This is the thin veil that everyone hides behind
There are many places in society where we do this. The thin veil of politeness, lies that everyone knows are lies – like asking "How are you?" without actually caring how the other person is, and answering "Quite well, thank you." when you're neither okay nor thankful.
It makes no sense, but people would behave worse without the illusion. I know not whether the benefits of transparency would outweigh that.
You expect a first year junior grad student (e.g., me before i was inured to it) to blow the whistle on their senior, well respected, famous, advisor? Is this a joke? You think change like this happens bottoms up from the indentured servant^1 class? lol.
> It makes no sense, but people would behave worse without the illusion.
wut. we've taken a large step off the sane road here. democratic/fair/equitable processes are open. full stop. if you want to debate me on this you're gonna have to whip out a whole lot of philosophy, history, data, etc.
^1 do you know how many phd students in the US are foreign nationals on student visas? if you had to pick between majority and minority which one would you pick? hint: you're wrong it's the other one.
> democratic/fair/equitable processes are open. full stop.
Secret ballots. Double-blind medical trials. The sealing of juvenile court records.
It's not as clear cut as you make it out to be: what might make sense in one academic field (e.g. applied particle physics) might be total nonsense in another (e.g. partial differential equations).
in the first and the third you're conflating the processes of the governing body with the experiences of the governed - the governed need to be protected from the governing body, not the other way around - that is the whole point of demanding that the governing body operate in the open. like bro politicians (as politicians) have no right to privacy, but we as individual citizens, governed by them do. how is it possible that i need to explain this to you?
the second isn't a democratic process of any kind.
> It's not as clear cut as you make it out to be: what might make sense in one academic field (e.g. applied particle physics) might be total nonsense in another (e.g. partial differential equations).
oh but it is and you'd have to be as obtuse as my reviewers to argue differently. like bro there isn't a scholar/thinker/activist on this subject anywhere in the world that agrees with you (politicians definitely agreew with you though).
> like bro there isn't a scholar/thinker/activist on this subject anywhere in the world that agrees with you
Not a 'bro'; also https://clauswilke.com/blog/2014/10/12/in-defense-of-anonymo... https://doi.org/10.1080/01411920802044438 https://doi.org/10.1038/249601a0 . I pretty much agree exactly with Claus O. Wilke, and haven't read beyond the abstract of the other two, but these were all on the first page of the first web search query I tried.
It's not well-documented, but even OpenReview permits anonymous reviews: see https://docs.openreview.net/reports/conferences/openreview-n...
> After the notification deadline, accepted papers were made public and open for non-anonymous public commenting, along with their anonymous reviews, meta-reviews, and author responses.
Your (incorrect) perception of an overwhelming consensus probably isn't why you hold the position you do. I'd like to hear your actual justification, if you would be happy to share it.
> Your (incorrect) perception of an overwhelming consensus probably
Did you know that the word consensus does not include the word unanimous? Just like there is a consensus on climate change despite some exceptional crank scientists there is a consensus on democratic processes being open - again i will emphasize that i never claimed there was consensus about whether peer review should be open but about whether government should be (and my own claims about peer review are informed by that consensus).
> Not a 'bro';
One of my favorite things is when someone is either so pretentious or so clueless as to reject a (gender neutral) colloquialism.
Yes. Did you remember that you wrote this?
> oh but it is and you'd have to be as obtuse as my reviewers to argue differently. like bro there isn't a scholar/thinker/activist on this subject anywhere in the world that agrees with you (politicians definitely agreew with you though).
If you would like to share the reasons for your opinion about academic peer review, feel free.
> One of my favorite things is when someone is either so pretentious or so clueless as to reject a (gender neutral) colloquialism.
Huh, so it is. https://languagelog.ldc.upenn.edu/nll/?p=48576 Mark me down for 'clueless'. I'd seen that with “girlie” and “guy” (perhaps coincidentally, both of which match historical uses of the words), but not “bro”. It's not listed in Wiktionary (https://en.wiktionary.org/wiki/bro#English), but it looks WT:ATTEST-able: do you have a definition that could go there?
This is so extremely common that I don't believe that there is anything to report. I reviewed dozens of papers under my PI's name throughout my entire PhD. Now that I have completed my degree, I never want to review another paper again.
If it's that dishonest, I don't want anything to do with it.
Thank you both for being honest.
my hypothesis on the value of a PhD: if you are the kind of person that can finish an honest PhD i.e., real work that's not necesssarily novel (I don't give a flying fuck about "novelty") but actually requires you to stretch hard to achieve, then you don't need one and it will cost you years of productive/rewarding/lucrative work in industry (and 100% academia is not for you).
i don't know what the "on the otherhand" is though - I really have no idea how anyone comes away from this process thinking "hmm yes I want more of this pile of bs" so I have no idea what kind of psychopaths go into academia (for which the PhD is indeed a prerequisite).
Consider another unethical practice: sexual harassment, in many industries, was once so extremely common that there was nothing to report. Everyone in a position to do something about it knew it happened, yet it continued. People reported it anyway. Tanked some of the reporters' careers, but the practice is way less accepted now.
You've completed your PhD, so you're at less risk of backlash. Are you yet in a position where you can blow the whistle on what happened to you?