Top academic institutions in 2018: normalized | Hacker News Reader

Top academic institutions in 2018: normalized

nature.com

57 pointseu17 comments

This is a fairly bizarre way to rank academic institutions. First of all, the methodology almost entirely neglects computer science because its metric of success is papers published in top tier journals; computer scientists tend to submit to conferences (e.g., NeurIPS) and hence get no credit for their work in this count. However, other fields seem over-represented from the list of journals---this is likely why Cold Spring Harbor, a very good biological lab where probably the vast majority of the papers are published in Nature-approved venues seems to be so elite.

The "normalization" they use divides the proportional count of authors that have contributed to an article in the "Nature Index" to the total output of the institution in the sciences, measured by a company called Dimensions. This has the odd effect of penalizing institutions for publishing outside their listed journals.

Finally, as an academic, there are some journals on the index that I have published in, but many venues I have published in did not make their cut. Sometimes more specialized journals are necessary---one cannot easily publish, for example, a detailed proof of a theorem in Nature, even if the result is very important.

List of journals: https://www.natureindex.com/faq#introduction1

I guess you can thus say the list suffers from selection bias and skews small samples which makes it pretty useless but good marketing.

Although it's unfortunate to say, but when one makes a big scientific journal, the inclination is to go to one of the top journals (Nature index) before going to journals outside of the listed journals. The reason behind this is that getting a paper in one of those journals is directly related to academic payoff (prizes, promotions, grants). Viewed from this payoff perspective, reason behind this Nature index isn't so unreasonable.

This Nature index would be equivalent to counting the number of papers accepted into the top computer science journals. So yes this index ignores the computer science discipline.

The lists, and Nature index, seem like a pretty transparent ploy. Want your institution ranked well? Make sure you publish with us.

Departments and institutions already fret over US News rankings. I wonder if we'll see memos encouraging publication in NPG journals if these rankings become regular.

I was comparing their #1 Cold Spring Harbor Lab to HHMI Janelia Research Campus, which doesn't even seem to be ranked in the top 100. Yet when I look at the research output, it seems pretty similar with the nod to Janelia in terms of FC, top article Altmetric score and article count.

https://go.nature.com/2x8zE68 (Janelia Nature Index FC = 36.41)

https://go.nature.com/2RsQl5n (CSHL Nature Index FC = 31.24)

So the denominator for the # of articles per Dimensions must be much higher or N/A and disqualifies some institutions? Unfortunately the Dimensions article count isn't viewable for any institution not making the top 100 list.

As an Austrian, I‘m very happy that the just ten year old IST Institute made it to second place. What really sets this institute apart is its lack of departments and thus its organic multi-disciplinary approach and culture. An approach I personally find very fruitful in my social relationships as well, as I learn so much from people who do different things than I do. It‘s enriching in more ways than I imagined.

As a non-Austrian doing fairly well at a top university, IST has put Austrian academia in my list of possible next stops and far higher than much more prestigious places.

From what I've heard from some fellows, they are doing a really good job at creating a nice academic place sans all the bullshit that comes associated with that lately i.e., politics. Probably a fresh start helped.

I wish you all the best!

I'm uncomfortable that number of papers was considered part of the quality of the institution as a whole...

It was published by Nature. The number of papers is the only metric they care about.

One potential concern with this normalization is that it could boost institutions with a single world class research group. That group could be pumping out papers while the rest of the school is average.

Not at all surprised to see Brandeis on there.

Is Brandeis supposed to be that good ?

I’ve just finished reading “DNA: the story of a genetic revolution” by J. Watson, Nobel prize winner, who was a long time director of Cold Spring Harbor laboratory (number 1 on that list). It is an awesome and very easy overview of the field which I highly recommend (one of the best books I’ve ever read)

Watson is a very good writer - and funny.

I've heard a lot of unsavory things about him but I've also heard that his unique perspective led directly to the climate at CSHL which positioned the lab to be at the center of so many breakthroughs.

more importantly than a good writer, he is great scientist and scientific leader. It takes a special person to create and/or lead the growth of a team/lab/institution to world-class level. Somewhat reminds me of M. Kelly from Bell Labs

I wonder if a calculation similar to the h-index measure could be used rather than giving each submitted paper equal weight.

Reply on news.ycombinator.com