Comment by impendia | Hacker News Reader

Comment by impendia | Hacker News Reader

> Increasingly it will be seen as perverse to submit a paper to a journal and wait 12 months for comments from two scientists, instead of sharing it on a platform like Academia.edu and getting comments from hundreds of scientists in two weeks.

This seems absurdly farfetched for me. It is easy and common for mathematicians to share their work (e.g. on the arXiv) and I consider myself fortunate to get two or three good comments, let alone hundreds. (Of course, this is a claim made by TC, not by academia.edu.) Whatever the virtues of academia.edu or any other technological solution, I do not expect it to multiply the audience for my work by an order of magnitude.

By way of comparison, check out Terry Tao's math blog. His reputation, his expositional ability, and the size and enthusiasm of his audience are surpassed by no one. And of course, the blog (and the ability to comment) are freely available to everyone with a web browser.

http://terrytao.wordpress.com/

Look at the comment counts. For many posts, in the single digits. And this low, even though Tao personally responds to many of the comments and questions.

I suggest that this is likely to be an upper bound for responses to math writing posted anywhere, in any format, with or without restrictions on how can read, write, or respond to what's posted.

I don't think peer review is broken, at least not in math. It can be annoying to do (which is why it takes so long), but we regard it as an obligation and that's why we do it.

The issue, in my mind, is to separate peer review from "publishing", or more to the point, from dissemination. The reason this problem is so thorny and obnoxious (again, in my mind) is that in principle it is trivial.

I believe that the two problems you mention, about peer review taking so long, and mathematicians not generally commenting much online, can be solved with reputation metrics.

The reason scientists don't blog a lot, or comment on blogs, is that there aren't reputation metrics associated with that kind of contribution to the scientific canon. It is the same reason that scientists don't share their data-sets. It is work to blog, and to prepare data-sets for public consumption, and there is no reward. If we reward scientists for sharing their comments on each other's papers, and for sharing data-sets, more scientists will be incentivized to make these kinds of contributions to the scientific canon.

The same goes for speed. When the journal editor asks a scientist to peer review a paper, the scientist accepts the request out of a sense of social obligation, but immediately that job goes to the bottom of her list of priorities. She will get no credit for doing the peer review, and so there is no reason to do it quickly.

This de-prioritization of peer review slows down the whole scientific communication system. To speed it up, we need to incentivize scientists to share their comments quickly. If someone sees something wrong the description of an experimental set-up, the system needs to be set up in such a way that they are incentivized to share that insight quickly.

I've been saying for many years - if scientists can't break past the issue of copyrights, nobody will. Academia concentrates probably the smartest minds on earth. They should do better than cowering under the tyranny of copyright. They should lead the way.

Few realize that signing over copyright to the publisher frees authors from protecting themselves in case of infringements on their creation. Few also realize the marketing and indexing power of the publisher. Break away from copyright? "Be careful what you wish for", because you're on your own from then on.

> The reason scientists don't blog a lot, or comment on blogs, is that there aren't reputation metrics associated with that kind of contribution to the scientific canon.

Scientists often present their unpublished data at conferences, even when the presentation has little prestige, and in fields where citing conference abstracts is rare, so the effect on one's reputation can't be quantified.

The reason that most scientists don't blog or comment on blogs is that they don't perceive these activities as contributing to scientific progress. This might be because these aren't a good format for presenting original research, because appropriate publishing platforms don't exist, or simply their fellow scientists wouldn't read their blogs. This is quite different from peer review, which most scientists recognize as an integral part of the system.

It makes little sense to create a "reputation" metric to index contributions that the scientific community does not believe are meaningful. First, focus on figuring out why scientists don't think blogging is science, and the rest will follow.

> The reason scientists don't blog a lot, or comment on blogs, is that...

So, now that I've read the comments on a few of these open-access type articles, it seems like there is so much variation among academic disciplines that broad statements about "scientists" are untrue in many cases and probably isn't a helpful way to phase things. This statement is not true in every field: in Economics (my field, you can decide whether or not we are "scientists," but we certainly try to publish in peer-reviewed journals) there are many blogs by extremely accomplished and prominent Economists. They get a lot of informal but meaningful credit for it, in some cases much more than another top publication would bring (when you go from zero to one or one to two top-tier articles, it affects your career. When you go from 31 to 32, not so much).

Just for specifics: Jim Hamilton at UCSD (my alma mater) started and contributes to the blog http://www.econbrowser.com. His graduate students started landing order-of-magnitude better jobs and interviews once the blog was established (and, just to make clear that it's not sour grapes on my part since most of them are my friends: these are jobs that the students are quite well qualified for; I think previous students if anything got too little exposure). It's a great blog that's really well regarded among Economists, and everyone I've asked thinks that the blog is bringing him a lot of (IMO deserved) attention.

> This de-prioritization of peer review slows down the whole scientific communication system.

In any field that I'm familiar with, scientific communication does not happen through journal articles. It happens through conference presentations, seminar presentations, informal discussion at conferences and when visiting other research institutions, working paper dissemination, and (somewhat, but growing) online.

A lot of that activity becomes formalized by comprehensive letters of recommendation that are solicited for every major promotion (in academia, obviously), so it gets heavily rewarded and is taken seriously.

Peer review exists as a check on those steps to make sure the impressions are right. Polish up the arguments, have an editor and referees sign off on it, and then put down a copy for the record for the people who need it to look over carefully.

If there are specific fields where that is not true, please spell them out.

I also thought that the time lag was a non-issue for math and physics since the advent of the ArXiv. Does anyone know why more fields don't participate, or create their own version of the ArXiv?

(I've heard mumbling about how biology is different because of proprietary issues, but it's never been clearly explained to me.)

In the medical field investments are on the scale of billions to bring a new drug to market, and each invention in part is somewhat independent.

On the other hand, in CS, algorithms are cheap to code and combine much more, it's always a fusion of many algorithms and patterns, so, putting restrictions on them has a chilling effect. Also, the perverse idea that adding "on the internet" to any previous idea makes it new and worthy of a patent.

Reply on news.ycombinator.com