Hacker News Reader

Comment by gjulianm | Hacker News Reader

At the risk of sounding like a Reddit comment, what's the difference? I've had reviews on papers of people that didn't even skim the content. I've had rejections from people who barely knew anything about the topic. I've had reviewers insist we should cite certain papers (funnily enough, all from the same lab even though they weren't really relevant).

The peer review process is absolutely broken. There are no incentives to do good peer review, there are perverse incentives to stop "competing" papers from being published or modified with convenient citations. Adding ChatGPT to this process doesn't really change anything. It could even improve it: it'd be a crappy reviewer but at least you can test yourself what it's going to say so you can avoid the 6-12 months waiting period for a review.

View on HN

I'm an editor at two journals and have been guest editor at others. I've reviewed countless papers.

I think your observation isn't too far off actually. As a reviewer once I saw another reviewer (in the editor's decision letter) give as citations some papers that I was supposedly a co-author on. The problem is, those papers never existed; they were some weird amalgamation of citations from actual papers with my coauthors. They were exactly the sort of thing you'd expect from generative AI today, but at that time (to my knowledge) it didn't exist. I always assumed the reviewer was tired and miscopied something, which they probably did, but as you point out, it's hard to tell sometimes and sometimes it almost doesn't matter given the other things that go on. The thing that you're pointing out is that this sort stuff still goes on, and has been going on, even without ChatGPT. In this weird kind of meta-process, it's as if using ChatGPT to mimic citations is itself doing the sort of thing ChatGPT itself does.

The thing no one seems to be mentioning is that the reviewers don't make the decisions, the editor does. The reviewers are supposed to be advisors or juries or something like that. There's an important discussion to be had about editorial quality at many journals, and I agree that this ChatGPT episode is just one illustration of the myriad ways academics is fundamentally broken, but it's an important thing to keep in mind.

Yeah I am literally about to review a paper which I definitely don't understand well enough to review. I will probably just write something very vapid and brief. Half tempted to use GPT, but I'm a good enough writer that I can make it up.

As opposed to being honest and trying to understand the paper, and if you fail then writing in your review that it is too difficult to understand?

Unfortunately both my mental health conditions and time constraints are such that I don't have the ability to spend time to understand the paper right now

Then don’t review it. I spend months or years writing papers. A voluntary reviewer who won’t bother reading is not helpful.

But then the journal may flag me as being unreliable. No thanks... I think I will just put in the shitty review

But you are unreliable. Selfish people like you makes the world a worse place.

Sorry but my career is an astronomically higher priority than any ideology of good peer review practises, and also an astronomically higher priority than "making the world a better place"

Oh now I remember your post about your depressing post doc. Why do you care about getting marked as an unreliable reviewer if you are not staying in academia after the post doc? If its for your cv you can just write reviewer for what ever journal you want. Nobody is gonna check it.

In Germany recently a medical specialist was jailed for 7.5 years for delivering wrong all clear results when reviewing skin cancer samples. He was aware he was not mentally healthy and was mot able to do his job.

Please get help instead of playing pretend and barreling through. People will get hurt by wrong expert statements. You fear for career while at the same time putting the career of another at risk.

People get respect for saying ‚no‘.

I'm a quantum physicist, not a medical doctor.

You have to learn to say no to reviewing tasks that you can't do.

That's not how having a job works

I get that you're stressed out and overworked, and I don't mean to add to it, but you seem to be confused between having an academic career and killing yourself. You're doing authors, editors, and yourself no favors by taking on load that you can't handle.

Have thought about stopping being a reviewer?

Reviewing isn't an occupation. I just get endless emails badgering me into accepting to review papers. If you say you can't, they demand a suggestion for someone who knows more. The problem is, there aren't many people

It goes without saying that it isn't an occupation. You are not helping anyone by doing sub low quality reviewing. It would be better if noone reviewed than getting low effort reviews like what you are describing doing.

Unless it's specifically part of your jobto do so, you have the right to say 'no' to these emails, especially when you're already struggling with your mental health. Prioritize your own wellbeing over people who are so entitled that they make demands that are uncalled for.

Reply on news.ycombinator.com