Hacker News Reader

Comment by impendia | Hacker News Reader

> Third, MIT should instruct its deans and other officials to no longer look favorably upon the mere fact of publication in a “prestigious” journal when making hiring and tenure decisions. Instead, promotions should be based on the quality of a person’s work, wherever it’s been published. (This sounds obvious, but most people in academia will tell you that where you publish is just as important as what you publish.)

I wish I liked this article, but it seems that Manjoo has failed to appreciate why this problem is difficult, and therefore, why it has not been solved already.

View on HN

Yeah, the casual suggestion that MIT should try to badly damage the careers of most of its faculty (presumably causing them to leave) suggests that he hasn't thought about this problem very long or talked about it with many people. As much as I dislike academic publishing, the last thing I want is to have deans and their staff evaluate the merits of research on a case by case basis. It sounds nice in theory, I guess, but they're people with agendas and biases too, and even setting that aside, they don't have the time or the training to do it. (as an, I hope, neutral example: could a dean trained as an economist really evaluate the relative merits of 15 different experimental physics papers? Could a trained physicist really evaluate the contributions of 15 different economics papers?) (note that this all applies to deans at other universities. The deans at my university are of course exceptional and above reproach).

So, on the one hand, even MIT probably can't unilaterally make a lot of progress on this issue. On the other hand... almost every recent paper I've ever been interested in is available from the author's webpage (making this quote: "much of the work produced in academia is never seen by anyone outside that cloistered world, because everyone who’s not affiliated with a university is cut off from access" ring hollow to me), all of the work to produce the journals is done for free or nearly free by researchers, and (anecdotally) everyone prefers electronic copies of the journal articles anyway. It seems like open access publication is almost inevitable.

A more manageable (but less sexy) step might be for MIT and other universities to defray the admin costs for some of these flagship journals that want to go open-access. The article mentions the American Historical Review as having annual (editorial) costs of about $500k [1], which is tiny for a university.

[1] From this post: http://blog.historians.org/news/1734/aha-statement-on-schola...

You're right that the central challenge here is helping academics evaluate each other's work without relying so heavily on journal titles, closed access journals or otherwise. There is some movement here. Increasingly hiring committees are taking other reputation metrics into account when evaluating work (inbound citation counts, readership metrics, follower counts - anything that may make the job of an evaluation committee a little bit easier).

The fact that papers you are looking for tend to be available on author's webpages is atypical. A very small percentage of the 2 million papers published each year is available online for free. Institutions pay $8 billion a year to subscribe to journals, and they would be only too glad to cancel those subscriptions if they could.

>The fact that papers you are looking for tend to be available on author's webpages is atypical. A very small percentage of the 2 million papers published each year is available online for free.

I'd love to see data on this, so I can see why my experience is atypical: are conventions different in some fields than others? Do I just have good luck? Am I misremembering or just wrong (which is possible)? The original article states, "Much of the work produced in academia is never seen by anyone outside that cloistered world, because everyone who’s not affiliated with a university is cut off from access" and I'd really like to know exactly which articles Manjoo has looked for and been unable to find w/out a university license. Institutional access needs to be a lot more comprehensive than what almost any individual needs access to, so I don't think that their subscription costs really are compelling evidence against my experience.

I'm kind of surprised that no one's pointed out an easy and really effective step MIT could take unilaterally: make all of the journals the MIT Press publishes open access (MIT is also a publisher). Some of them are among the best journals in their fields (e.g. the Quarterly Journal of Economics is one of the 4 most prestigious journals in econ and the Review of Economics and Statistics is very very good).

Why is this the top-rated comment? I could literally use it as a generic I-know-more-than-the-author comment on any article about a problem (replacing the quote and author's name of course). It provides no more information than that the commenter thinks there's something wrong with the quote that's pulled out of the article.

A good comment would at least say why you think it's a difficult problem (that the author has supposedly missed).

As far as hiring or giving tenure based on merit instead of journal prestige, this is achievable. It's hard because traditionally journal names are extremely important. But it's doable. When I was a grad student, I was personally aware of the process of hiring people in my field. The problem right now is that one person could say, "X is good but candidate Y has two articles in Awesome Journal, so Y seems better." That statement implies Y has good work, but we need to make room for the reply, "Yes, but X has done better work and avoided Awesome J. because it's closed access." It's an attitude shift, which is hard, but doable.

You objected to the upvotes, but do you actually disagree with me? I do not disagree with your claims. In your last paragraph, are you claiming that your suggestion is not only "achievable" and "doable", but simple enough to implement that the challenges don't bear mention in an argument for reforming academic publishing?

For elaboration, I endorse and agree with pseut's comment, he said more or less what I would say.

I recently served on a hiring committee in my own mathematics department. It is extraordinarily difficult to evaluate the work even of other mathematicians. In areas other than my own, I can distinguish bogus work from good work, but I find it difficult to distinguish good work from excellent work. Although we rely principally on letters of recommendation, and interviews (although we must of course first narrow our search down to a shortlist), we find that quality of journals is a good barometer for quality of work.

I would like to see publishing change as much as anyone, and I am a signatory to the Elsevier boycott (see www.thecostofknowledge.com). I believe we stand the best chance of effecting change if we seriously consider the obstacles standing in our way.

I agree that it's very difficult to change hiring attitudes to support open access publishing. I disagree that difficulty means it's a mistake to try.

Hiring/tenure attitudes can include respect for people who choose to avoid closed-access publishers. This doesn't punish people who continue to publish the old way. If done right, it only helps the careers of those who avoid closed journals.

By the way, I somewhat snarkily complained that your comment was snarky. That was hypocritical of me, at least in spirit. Sorry about that. I like non-snarky, straightforward comments.

I'm glad you're signed up at the cost of knowledge. My personal role in open access is mainly as the creator of that site.

The problem is that the quality of work in a related field can be hard to judge. However researchers generally try to get papers into the most prestigious journal that they think they can, and journals work hard to try to maintain consistent standards. Therefore the journal it got published in is a reasonable proxy for article quality.

> MIT should instruct its deans and other officials to no longer look favorably upon the mere fact of publication in a “prestigious” journal when making hiring and tenure decisions. Instead, promotions should be based on the quality of a person’s work, wherever it’s been published. (This sounds obvious, but most people in academia will tell you that where you publish is just as important as what you publish.)

This is an author that is confusing "most people in academia" with MIT.

Mere publication is not the path to tenure at MIT, and the tenure committees are well-versed in the difference between impact and the # of pubs.

Reply on news.ycombinator.com