Hacker News Reader

Comment by abstractalgo | Hacker News Reader

For manual flagging, you can use Unfold Research to leave a review on the paper itself for anyone to see it, and you can add review tags and a description about what you think is wrong with it: https://twitter.com/UnfoldResearch/status/156099536649284812...

Tools like scite (https://scite.ai/) do automatic processing (NLP) of other papers and determine whether a reference relationships supports or rejects the paper.

Also, http://octopus.ac/ did NLP processing on papers to make a breakdown of them and extract pieces to populate the content on their website, but they seem to have removed the content on their Github since then.

View on HN

Could manual flagging be abused by a country or agent against another country or agent in order to maliciously discredit papers from that target?

Most definitely.

An advantage of having open peer review system is also that this kind of attempts could be detected by tools doing automatic processing of scientific knowledge graphs generated by peer review activity.

This, coupled with verification of users, should be able to minimize the risk quite significantly.

Seems to me that this wouldn't be enough in case of state actors or large organizations. They would just have to employ real people to carry the fake review bombs.

Instead of a flagging system ranking the credibility of papers once for everyone, can we have a system where each user decides which other users they trust to evaluate papers, and ignore any signal from untrusted users?

For example, I could L1 trust people to evaluate papers and the trustworthiness of other users, and L2 trust others only to evaluate the trustworthiness of papers. If I notice a spurious signal getting in, I can find out who it came from and prune my list of trusted users.

I imagine I'm not the first to think of this. Anyone know of reading material on the topic?

The one thing I'm aware of is Brave's Goggles project: https://search.brave.com/help/goggles - it basically allows you to set up your own ranking algorithm with a specific syntax (it links to Github repo and, there, to a white paper).

I haven't seen more attempts at similar things, which doesn't mean they don't exist though. (There were mentions of a similar feature being added when Elon Musk was about to buy Twitter, but just gossips)

Sounds like journal clubs but organized through a specialized tech platform rather than normal communication channels.

(Journal clubs are groups of people, usually working in the same field, who pick & present a paper to the rest of the group on an rotating basis.)

You have also got https://pubpeer.com/

Reply on news.ycombinator.com