Hacker News Reader

While the parent is right in stressing the importance of the review phase, which is a filter to decide whether we will ever see a candidate paper or not, it is indeed the case, as the OP posits, that there is merit in extracting the context in which a citation is used, which has been termed "citation polarity".

For example, 'Chomsky (1969) was entirely wrong when he said that “it must be recognized that the notion of 'probability of a sentence' is an entirely useless one, under any known interpretation of this term".'

There are Natural Language Processing methods and tools to extract the citation polarity. Researchers like S. Teufel have used citation polarity and other methods to analyze publications ("rhetorical zoning") and to create maps of the published literature: https://www.cl.cam.ac.uk/~sht25/az.html

A scientific literature search engine (Semantic Scholar, CiteSeerX etc. - sadly Microsoft Academic has recently been discontinued) can benefit from such knowledge; for example, PageRank can be modified so as to incorporate citation polarity in the random walker model that underlies it. Think of it as adding a "dislike" button to a system that already offers a graph with "like" button relations.

The Microsoft Academic dataset has been picked up by OpenAlex.org

Citation rings are a huge problem, but couldn't the sentiment-analysis side be readily addressed? There's a difference between papers that just cite a paper without judgment (+0), papers whose results explicitly support another paper (+1) and papers that explicitly reject another paper (-1). Such an AI could also bypass "citation analysis" and instead ingest the entire scientific literature, and find refutations in papers that don't even cite each other.

In practice, 2 problems: 1) this requires sophisticated AI; and 2) many people will keep assuming that high citations = credible (i.e. assume that +0 and +1 are equivalent).

But this method will help researchers go beyond "it's highly-cited therefore I trust it"; it'll help students and rigorous scholars find how well-supported a paper is, despite its popularity. And it'll surface refutations that may be hard to otherwise find; for example when a scholar that's famous in a field, and unknown in another, decides to publish a paper in the latter field that may not be widely seen.

Ultimately (though this is far off: theoretically completely possible but highly technically difficult) we can also imagine AIs that ingest the whole literature, gains deep cross-domain knowledge, and are trained to detect poor methodologies, or automatically highlight new insights in one discipline that can enrich another discipline, or (ideally) make its own judgments about the merits of any finding based on the knowledge it acquired. After human review, it could systemically help "clean up" the scientific literature of bad methodologies and ideas, and free scientists to spend less time working on scientific dead ends.

That last paragraph won't be doable for a while, but the first 3 seem readily within reach of today's AI technologies (and seems to be what scite is doing).

I disagree that that is the sole (or even most important) point of a citation. Most of the time you can view a citation as part of the (extended, implicit) problem statement for the paper, e.g. here is what X did in this space, and we are different/better/build on that work by doing Y. If an important citation is missing, it shows you aren't familiar with what has been done before in relation to the problem you are trying to solve (bad). Of course reviewers and readers are free to disagree with your characterization of a cited paper in relation to your own.

Having said that, maybe if you publish in journals a lot this is more of an issue. In conferences the Program Committee is usually set in advance so there is less scope to target specific reviewers (other than those on the PC).

>My advisor taught me that the point of a citation is not to give your opinion on which other papers are good or bad; it's to tell the editors which other scientists should be on your paper's review committee

As a professional consumer of academic papers paying commercial rates for decades I appreciate your candor profoundly. Nevertheless what you say is a statement at least of disingenuity and in my opinion prima facie admission of the existence of a defrauded system.

That said, I have hope we can solve this problem outside the academic incentive system. Here's a relevant prototype developed by my research lab: https://braid.news/about

Thanks. I’ve wanted something similar to rank HN comments - I really want to read some users and I really don’t care to read the same tripe repeated by some other users.

I presume it is a hobby project but calling it PeeryView and then using braid.news was confusing to me. I didn’t even realise the “home” link would take me to the site (or maybe I didn’t see the home link, because I look to top left for that? e.g. Y logo on HN)

This is a very cynical and opportunistic way to do science. It may be the winning strategy to play the academic game and advance your career, though.