Web of trust for scientific peer review

code.google.com

5 points·mark_h··3 comments

This is a really interesting idea. A way of signaling quality of publications that is more useful than 'it made it through the committee at some conference/journal/etc' would be a major improvement for researchers.

The graph analysis is even more interesting - this is a way around the problem that publication decisions can be made based on reviews written by less-well-read grad students instead of the faculty one might prefer to hear from.

One question though - it seems like if this guy really wanted to sell the idea, he'd point us to at least one instance of it being used. All it'd take is asking someone to review an existing paper of his (or something). It'd be great to tinker with the review graph analysis, for example.

Why not even share a signed review of the project itself? I tried to generate one for this post, but it doesn't seem to work on OS X, so I filed a bug instead.

Exactly; the graph analysis was most interesting to me. The current situation is well beyond overloaded, and open to frequent gaming.

One attractive possibility if this took off is that conferences/journals could become less important as publication venues, and blogs etc could have equal weight. A conference provides a forum for meeting people, but I feel that most of the time the value of any venue is perceived solely in terms of its reputation (and hence impact on CV), and as you point out this is no guarantee that your paper has actually been reviewed by someone knowledgeable.

The project is an open-source command-line tool called gpeerreview. Great idea!

Reply on news.ycombinator.com