Hacker News Reader

Comment by bonoboTP | Hacker News Reader

People who submit to and review for conferences have known for quite some time how random that process is. Also they are slow. In a field that advances leaps and bounds on a monthly time scale, holding a conference half a year later or publishing a journal article more than a year later is just equal to irrelevance. The reason conferences still matter is because of legacy systems that are slow to adapt to the new reality (and because of networking opportunities).

But actually, thinking back, 10 years ago there may have been more of a stigma around a paper that was only on arxiv.

View on HN

If your result is not worth publishing in a journal a year or two from now, it't not worth submitting to arXiv today. Just write a blog post about it and move on. And maybe write a paper once you have discovered something of more lasting value.

What makes publishing it valuable? At best it's a second round of PR for the paper, after it was initially released on the arXiv. Reviewers sometimes have good suggestions, but you can get suggestions from people on your arXiv paper and post a v2.

From the perspective of advancing science, journals seem mostly pointless and most of their benefit could be had at next to no cost if we just agreed to publish on the arXiv and pair it with a review system.

From the perspective of advancing scientific careers, because of how founders and institutions treat arXiv papers, you have to still publish in a journal, and so people keep publishing in journals.

But like, at least in physics time passes at the speed of arXiv posts, and getting published usually happens after it's been cited many times already. And if people are already incorporating it into the web of knowledge/responding to it etc., then what did the journal do?

In my experience (in CS, bioinformatics, and genomics), the first round of reviews often improves the paper substantially. You are getting feedback from people outside your usual peer group, and they tend to notice things people closer to you don't.

And writing is thinking. Because you are forced to iterate on the text over a long period of time, your understanding of the topic improves and you learn to express the ideas better. Your job as a scientist is creating new knowledge, and the initial publication of new results is only the first step towards that.

It's worth publishing somewhere because PhD students need to graduate. And there are many PhD students because it's a ticket to industry labs. And because many professorships around popular topics were created. Those people need to graduate, so there have to be papers. Blog posts don't let you graduate. A good arxiv paper that just got rejected for unfair reasons, but otherwise gets a good community reception can still count towards your graduation if you have a sensible advisor.

Not sure if you understand how the sausage gets made. People won't spend effort at scale (there are always exceptions) on stuff that's not incentivized, such as blog posts. If you work on something, the goal is to make it count. But if you fail on that, arXiv is a second best option.

Also, papers that get published at conferences also get out of date pretty fast anyway. That's just how fast moving fields work.

But things may end up moving the way you are saying, as long as incentives keep up.

Journals (1-2 years) -> conferences (6 months) -> arxiv (weeks to write once the work is done, then days to publish) -> next thing (immediate AI writeup, code, results, tests, open scripts, weights, full reproducibility, full access allowed for other AIs to scrape and learn from, and reorganize and recombine on a speed of days or hours from result to adoption)

It's not been my experience that papers published in CS conferences get outdated quickly. Maybe AI/ML is different, but it's a weird edge case that should not dominate the discussion. Even CS as a whole is an outlier, because most fields that use arXiv don't have conference papers. If the expectations of AI/ML or even the wider CS are too different, maybe they could start a preprint repository of their own.

One of weird aspects of CS research is the tendency to publish small papers. In more experimental fields, a paper may represent years of work for the first author. But CS students often finish multiple projects every year. Maybe it's because of the focus on conferences. You want to attend conferences (to build your networks, and because it's interesting and valuable), but you often don't get reimbursed if you don't have anything to present. Students from other fields can just present their work in progress, but CS students have to come up with publishable results.

Maybe CS should also strive towards bigger projects. Instead of publishing individual results (which someone often improves a year or two later), the papers could focus on the outcomes of entire research directions. That way you are more likely to produce something of lasting value. Something that is worth writing up and rewriting and polishing, until you understand the ideas much better than after the initial write-up.

> Maybe AI/ML is different, but it's a weird edge case that should not dominate the discussion

I wonder about the recent proportions. Growth in ML papers has been mind boggling and I wouldn't be surprised if a major part of Arxiv's submission spike was in AI ML topics.

> Maybe CS should also strive towards bigger projects.

The current rule of thumb is that you need at least three accepted papers in good venues to graduate with a PhD. Until that changes, the incentives point towards trying to publish often. And PIs want this because they also get evaluated in this publish or perish system. This is my main point. Throttling is a stopgap solution because academia doesn't want to admit its internal problems that have been boiling over already before AI.

Reply on news.ycombinator.com