Ask HN: Why do research authors publish before supplementary material is ready | Hacker News Reader

Ask HN: Why do research authors publish before supplementary material is ready

15 pointsHard_Space7 comments

I write about AI, and pretty much every day have to write an email to the listed lead author of an Arxiv paper about the location of supplementary material that's mentioned in the paper, but not linked to in the abstract or anywhere else.

Sometimes the supp. material (such as a video) is so weak that I can see why the authors would prefer to engage readers before showing proof, but very often, the material is good.

Sometimes the supplementary video never actually comes at all (and I won't even bother asking about the 404 errors to listed GitHub repos, or the fact that they are placeholders 80% of the time).

Can any researchers here enlighten me about these practices? I had thought to try the 'No Stupid Questions' Reddit, but I think it might be a bit marginal for that crowd.

View on HN

Because publishing supplementary material mostly hurts researchers. As a researcher, you care about getting papers published and accumulating citations, and not much more. Saying that you will publish the material makes the paper look a bit better and legitimate to readers and reviewers, even if you never get around to it. Though to be fair, a lot of venues these days require to submit/publish at least the source code.

Some drawbacks of publishing materials such as code:

- You get random people asking questions about it, taking time away from your research. Not a problem if questions are from fellow other researchers or experts and lead to interesting conversations, but most questions are low quality aka "how do I run this on Windows!??"

- You risk people finding a bug in your research code, making your results invalid. Great for science. Bad for your citations, publication chances, and personal resume.

- You need to clean up / maintain code. Often, there is a bunch of private/company infrastructure stuff mixed with research code. Again time that could be better spent on the next paper.

There is basically no benefit to publishing materials other than Twitter clout and PR. If you think your paper/code can go viral due to the extra materials it's worth it because it leads to more citations.

Also, from my experience a lot of the time it's simply due to loss of motivation. The same reason why people abandon side projects. Researchers typically work on multiple projects, and when a paper is published they are likely already working on their next paper. That makes it very easy to procrastinate on extra materials for the "old project"

If you upload the source code first, someone else might use it while you wait for the months that the paper publication process takes. That's a risk if you want to claim your idea as new.

That's no excuse not to update arXiv, though. They have a form to submit code and supplemental URLs after publication. They also allow you to post a new PDF revision with supplementals attached.

Some conferences require that you publish your code. So all they have to do is add to the paper "our code will be published at this url: ..." and then once they're accepted they never do it.

Similarly supplementary material makes their paper seem better and more solid than it is, so they say they have it then never publish it.

It's just generally scummy behaviour from people who are under a lot of pressure to publish.

evaluation and the peer review process all pretty much ignore the supplementary materials, so leaving them until the end is kind of systematically encouraged.

> I won't even bother asking about the 404 errors to listed GitHub repos

Usually this is because the repo is private (rather than because it doesn't exist).

Sometimes this is a genuine mistake: people may use a private repo because they're concerned about a competing researcher seeing their work and scooping them, and intend to make it public once their paper has been accepted but forget.

Because people who distribute funding don't read your papers - they wouldn't understand them -, don't look up any supplementary material, they just count how many papers you've published.

Because conferences have submission deadlines. :)

Reply on news.ycombinator.com