Comment by NotOscarWilde | Hacker News Reader

Comment by NotOscarWilde | Hacker News Reader

Parent Full thread

I will comment on it from my point of view (theoretical CS PhD student):

> What's wrong with giving other researchers the ability to rate and comment on a paper.

I do not see nothing inherently bad with this (provided very good moderation tools are in place). However...

* > I would argue it works just as well a this journal reputation system *

As a journal reputation system, possibly; although I find that experts have often very different taste compared to say general Hacker News public -- for instance, machine learning has a very high "coolness factor" these days compared to other areas of computer science, but it would be unwise to make a system where a weaker paper on machine learning gets vastly more credit than a better paper on say approximation algorithms.

However, an important part of the journal submission process is the fact that the editors ask the members of the academic community for spending several days on the paper, noting major and minor details and checking the validity of the proof.

Since the reviewers do their work for free, there is no reason why there cannot be an open system that replicates the same process; still, just "adding a comment system to arXiv" will probably not be enough, as people spend rarely that much time on comments (15 minutes tops, not entire days).

Why do reviewers work for free? And why scientists agree to give away their work for free?

I think it's because no one can agree on a price, tax and international payments is too expensive, and there's a tit-for-tat understanding.

First, the price. The last paper I reviewed took me about 10 hours. I charge $200/hour. Should I charge $2000 for my time? If I'm paid by the hour, who tells me when to stop? If I'm paid a flat fee, how much is that fee?

I have seen a reviewer write something like "I read the paper twice. It seems good. Publish it." Does that get the same as someone who spends 10 hours for a detailed critique?

Second, does payment constitute work-for-hire? Does the publisher and/or reviewer need to pay taxes? Might some reviewers have a contract which prevents them from taking on part-time work? What if the reviewer is overseas? Do tax agencies from two different countries need to be involved?

Third, most people expect to publish, and get their papers reviewed. If a single author has three reviewers, then that author should expect to do about three reviews. This is an imperfect system which rewards publishing rather than reviewers, but there are also some compensations to the reviewer.

Moreover, I recall one story - I don't know the validity of it, but it has a ring of truth - where a daycare got annoyed with parents who picked up the children late. They decided to fine parents should that happen. As a result, children were picked up late more often because the fine was interpreted as a fee for extra daycare, and no longer social chastisement.

I suspect that reviewers are in a similar boat. I'll feel different about it if I get paid a pittance than if I were to do it for free, so long as others do the same for me in the future.

Your recall is correct - it was an Israeli daycare: http://priceonomics.com/effectiveness-of-fines-for-late-pick...

Reviewers don't really work for free; it's generally accepted that reviewing is part of their salaried job.

For free in the sense that the journals pay them nothing. So lets recap; the public/university pays for the researcher and the research, the public/university pays for the reviewing and submission process, and finally the public university pays to access the work they already paid to create and review. And people wonder why everyone but the journals think the journal process is broken.

Reviewers work for free for a few reasons reasons. First, it's prestigious to be a reviewer for a top journal and very beneficial for getting tenure. Second, it allows you to create connections with other top researchers. Third, it's giving back to the community by doing a service that needs to be done.

How do people know that I have (or haven't) been a reviewer for a top journal? Do they have a public list of reviewers?

Of the people I meet at a conference, how do I figure out who has been a reviewer for Nature, and how do I verify that claim?

My thoughts exactly. I've never known a journal to do anything to promote the fact that someone has reviewed for them (eg produce a letter on request that states which articles someone has reviewed for them). It'd be a nice and free way of creating value to reviewers who generally work for free, but I don't know any journal who's ever done this.

Reply on news.ycombinator.com