Comment by screye | Hacker News Reader

Comment by screye | Hacker News Reader

Imo, it has to do with incentives and rewards.(in CS at the very least)

The Hindex is a measure thats become the target. So 1 seminal paper is much much worse than 10 completely forgettable papers. It is also common to divide seminal work into smaller publishable morsels to rack up the hindex at the cost of novelty and clarity. The lack of incentive to pursue novel works, also means that most papers are incremental by design. Imo, this is the biggest waste of money at academic phd programs. However, the students need to take their career somewhere, so an uncomfortable compromise is met.

Another problem is how every conference is converging into the same impact maximizing mush without any meaningful differences between them. This has massively affected searchability, which as we know leads to even greater 'agglomeration at the top'. Having different standards for novelty, experimental rigor, math rigor, scale, technical fit and the like would allow for accepted papers to follow internally consistent searchable constraints, while the diversity of target audiences accross conferences would allow for different types of research to coexist. In my field, conferences are only differentiated by deadlines and status. Everything else is secondary.There arent too many papers. There just that the quality of curation has gone to shit.

The bigger problem is structural. Academic research is mostly done by early-career researchers in short-term positions. When you are a PhD student or a postdoc with only a few years to do research, you don't have the time for long-term projects. And because your career prospects are determined by your success in your current position, you have to manage the risks by working on multiple projects. By the time they graduate, the average successful PhD will then have several publishable results where they were the primary contributor.

There are very few positions for mid-career individual contributors in the academia. After ~10 years, you are expected to take a leadership position. Instead of doing research yourself, you are now expected to convince your peers that your ideas are worth funding and that they should collaborate with you. If you get the funding, you will hire people to do your research. Your job is then guiding them through the early career. You can start thinking about ambitious long-term projects, but in order to make them viable, you have to break them down into smaller publishable subprojects suitable for your students.

You accidentally conflated "conference" and "paper" which is true in computer science and also relevant to other fields. The crux of the problem is that almost no one reads papers. People get exposed to ideas at conferences, and conference organizers invite the people that wrote the seminal papers many years ago to be the keynote speakers. Even if a paper is cited, it is usually because they saw the author speaking at a conference.

It's painful because (IMHO) the H index is just a much worse approximation of something that we could actually achieve with PageRank for academic citations. In that case, a bunch of middling papers would be rewarded, but so too would one critical paper that lays a foundation for a field.

I always thought that PageRank was inspired on methods previously used for scoring academic papers. Now I'm wondering if I misunderstood something, or my professor misunderstood it first. Damn.

PageRank has nothing to do with h-index to my knowledge. The problem with using PageRank-like methods for academic papers is that academic papers mostly reference backwards in time (except for the occasional draft or work-in-progress being referenced, but proper cycles are rare). A triangular matrix doesn't yield an interesting stationary distribution...

You could just make it bidirectional (add the transpose and make it symmetric), right?

Too easy to game by citing the most reputed sources. Reputation shouldn't flow up a citation.

I don't know if it makes any difference, but if PageRank should replace the h-index, the most direct equivalent would be to rank the authors rather than the papers, no?

Wouldn't betwenness centrality be better?

Think of an army randomly moving through your citation graph; the more particularly nodes are trampled over, the more pagerank it has.

Now: if this army is informed about the shortest routes and instead moves about optimally, the most-trampled over places have higher betweenness centrality. I'd like my simulated citing scientist to be smart.

I've tried applying PageRank to pubmed with reasonable results, but betweenness does sound compelling. Thanks!

I'll give you a bonus: the Louvain algorithm for community detection. Whatever ships with Networkx, Gephi, etc. doesn't work for my (correlation-derived, pruned with graphical lasso) networks, but the Louvain method (a greedy approximation to modularity maximization; the real math magic is in the concept of modularity and the configuration model) is awesome.

Interesting. We use Louvain for single nucleus sequencing cell type identification. Cool to think about its other applications. Thanks for sharing.

The incentives in academia are to publish and receive citations. This is often very political and citations are made as a favor or trade, completely independent of the research.

Unfortunately this structure forces out productive scientists who want to do interesting research rather than churn papers.

Doesn't h-index prevent exactly the scenario of having 10 forgettable papers? Having a couple of great papers yields a high h-index, but 10 forgettable papers would hold the number at a low count because you need N papers with at least N citations each. So a seminar paper would +1 to that h-index indefinitely, whereas low value papers would upper bound the h-index to their citation count.

It kind of regresses to a median. 1 big paper, and 100 papers with zero citations aren't that useful.

However, most top phd students/ assistant professors hover around the nebulous 5-30 hindex where getting 30 citations is a lot easier than publishing 30 papers. So, in most cases, you will prefer to figure out quantity, because the quality bar is so low. Additionally, they and lab mates always cite each other which leads to a free 10-ish citations overtime anyway. Lastly, authorship priority is not taken into account in hindex. So, a bunch of secondary-authors can easily get those numbers up at massively industrialized labs. So a small set of productive 1st authorships are given lower weightage than a large list of low-contribution 2nd authorships. Almost all super-high hindex professors are more like CEOs of a research company than primary researchers.

H-index, like all metrics is useful. It sort of shows the median quality of papers by an author assuming that equal time is spent on all papers. It is informative, but making it too important in academia has led to it getting gamed with counter-productive incentive structures.

H-index ignores away a lot qualities that are incredibly important to being a productive researcher, and has led to researchers with such qualities being progressively pushed out of academia ever since it has become THE target.

> assuming that equal time is spent on all papers.

And that's exactly what makes the h-index useless, every paper has a different effort put into it. I've had papers that required 5y of work, some others only 1 month... And yet they are counted the same.

Also the number of citations depends on the field, for people working across multiple fields you get papers that are highly important for a smaller community get less citations than low quality papers in a large community... Most papers are cited "by chance". Researcher type two keywords, cite paper that seem to go somewhat toward the wanted direction, rinse repeat.

> Lastly, authorship priority is not taken into account in hindex.

Given that there is no standardised way to determine authorship priority - given that this doesn't even make sense in plenty of cases - I really don't see that as a problem.

Apparently, some folks divine some nebulous properties from author order. Others just put authors in alphabetical order.

If you want to know who contributed what, ask the authors. You may get different answers from different authors, which is emphatically not a flaw. (In contrast to the aforementioned divination procedure.)

Reply on news.ycombinator.com