Comment by schlipity | Hacker News Reader

Comment by schlipity | Hacker News Reader

Parent Full thread

Forgive me for not being an academic, so maybe this question is moot.

Why isn't there a place that links to a given paper so that discussion about the paper can be centralized? It could also contain links to papers that link to that paper, among them would/could be the failure to replicate information, adding to the discussion. And I don't really mean a topical "this is what's new" site, I mean a historical "This is the paper, and this is what people have said about it." sort of site.

This seems like a fairly elementary idea. The only seeming difficult bits I see are:

a) Getting (legal?) access to these papers.

b) Dealing with a large number of papers (millions?).

c) Authenticating users to keep the discussion level high.

d) Moderating the discussion in a way that doesn't piss off academia (impossible?).

e) Keeping the number of these sites (competition, if you will) low so that the discussion is not fractured between them.

It would seem like one of the "Information wants to be free" sites that host the papers that everyone shares with each other would be a great place to start something like this.

In general the way academics have a “discussion about the paper” is by writing another paper in response and citing the original.

So what you’re looking for is the citation graph, and you can find it e.g. on Google scholar. Search for the paper you are interested in, and then click the link which lists all the papers which cite that one.

You’ll probably need to be on a university campus or ask your anonymous internet friends for help to get around paywalls.

That only gives you academic critique and follow-ups. Sometimes, some practitioner is pissed to the degree he/she posts an opinion online. That is extremely valuable, but never gets into the citation graph. For example... http://bramcohen.livejournal.com/20140.html?thread=214956

> I'd comment on academic papers more, but generally they're so bad that evaluating them does little more than go over epistemological problems with their methodology, and is honestly a waste of time.

Yes, we write comment papers. But the process is very very slow, so many don't bother.

I submitted a comment paper to a journal in March last year. The paper I was commenting on was from November 2014. My comment got accepted, after two rounds of peer review, in May this year. It will appear in the journal in July some time.

And that's fast by academic publishing standards. A flamewar that would take a day in an internet forum can smoulder for decades in academia, while the principals gather supporters and snipe at each other in conference sessions.

> You’ll probably need to be on a university campus or ask your anonymous internet friends for help to get around paywalls.

Sci-Hub is now everybody's internet friend.

This is exactly what ResearchGate aims to be - a social networking platform for scientists and researchers to share papers, ask and answer questions, and find collaborators.

In this interview [1] Ijad Madisch the founder & CEO of ResearchGate envisions ResearchGate to be a home for negative results.

[1] http://www.zmescience.com/other/interviews/qa-with-dr-ijad-m...

The idea is good, but the execution is LinkedIn-level horrible. Dark patterns everywhere, useless gamification which promotes low quality content, etc.

Yeah research gate has some of the most awful dark patterns/invitation spam of any social networking site. Co-authors constantly inviting you to join, convincing you people that are not on the site are on the site, etc...

a) Maybe just have Sci-Hub and/or ArXiv links? If you assume your audience is mostly professors, then they can get past paywalls and you can just link to a paywalled source.

b) Nested subtopics, good search, and mergeable threads so you can enforce one thread per paper? Do papers have ISBN numbers, or some equivalent?

c) Most professors have verifiable .edu email addresses, but this is going to take human effort and you'd have to decide how much of the .edu world you want to allow in.

d) Start with a very specific code of conduct, accept the fact some will be pissed off, have at least one full-time paid moderator? This will be tough, maybe the hardest part.

e) Network effects will do this for you within networks (so each subfield of academia will eventually migrate to one site, but you might be able to attract e.g. most of math but biology could be on a competitor). I would worry about this least.

I think this would work best apart from a paper-hosting site. The human costs and hosting costs will become expensive and it's easier to collect ad money if you stay legal. You might even able able to host this on a university network as ArXiv is, but this would make moderation more difficult because you'd be accused of bias towards that university.

I think PHP-BB has all of the technical features you would actually need. The moderation would be the hardest part IMO.

For (b) – yes, papers generally get DOIs.

I would love to have something like this. However I think that you may be overestimating the willingness of people to contribute high quality discussion about a paper. The people best able to offer expert critiques are the same people who have the least time to do so. The fact that the peer review system regularly manages to do so is something of a miracle, but that's because we've managed to get prestige and career advancement tied to membership on program committees and editorial boards.

I think for something like this to work you'd need to think carefully about how to compensate reviewers. And I don't think that compensating them with money will work that well, since most academics care much more about career advancement and prestige than money (otherwise we'd be in industry...)

PubPeer does something along these lines already. The bigger problem is that there's not really any incentive to participate in this, beyond blowing off some steam and a vague sense of "service to the field."

Most important reason: a lot of potential reviewers wouldn't like that model for a variety of reasons.

Reviewing, generally, isn't prestigious or well-rewarded. So potential reviewers need to be kept as happy as possible. Internet forums have a lot of frustrating problems that IRL conversations don't have, especially for people who wouldn't otherwise use an internet forum.

You have to remember that the people who enjoy participating in written conversations online -- even young people who grew up on the internet -- is still a small fragment of the population.

f) Cost of maintaining discussion site. First the computer cost. Second the time labor of the moderators. Even with for-profit journals, much of the labour is free. Its based on the the pay-forward model: "I'll constructively review your paper for free hoping you'll do the same mine and others papers". But still every publisher has a paid managing editor to supervise the overall process.

Reply on news.ycombinator.com