Hacker News Reader

> Moderation for Arxiv is a silly idea. It already shouldn't be taken as a quality signal that something is able to be up on Arxiv.

It sounds like you're advocating for arXiv to become viXra. On https://vixra.org/all/2610 the second paper is currently "Correct Interpretation of the Great Discoveries in Particle Physics: I. Reconsidering the Higgs Boson via Vedic Vortex Structure", followed by a paper arguing that classical electrodynamics is bogus, "Gauss’s Flux Theorem Does not Hold in Time-Varying Electric Fields", and then "On the Boundary Problem in the Origins of Matter, Life, and Consciousness".

I don't think arXiv will continue to receive "quite some funding" if this is what it contains.

This is always the fate of explicitly "alternative" platforms, because only those people go there who are pushed out of the mainstream, who will tend to have some serious deficiencies, otherwise they'd try to be on the mainstream platform. Same with unmoderated social media turning into far-right places. Once a place obtains this type of reputation, it will be even more repellent for anyone not like that who wants to make it clear they are not like that, leading to a cascade, where the thing becomes a hermetically isolated "radioactive" place.

By the same principle, someone in the early 2000s could reasonably say "only weirdos date online", but the mainstream can shift also.

Vixra suffers from the "Witch Hunt problem" because Arxiv exists.

If Arxiv was the only game in town and it allowed everything, the result would be more like GitHub or Substack. That is, you wouldn't be able to trust everything hosted there, and a lot of it would be low-quality promotional garbage, with filtering moved to a different layer.

You could do some kind of karma system for example, where the prestige of your institution, your own karma, number of citations, papers published in high impact-factor journals etc would affect karma weight. You could have journals be like "awesome-x" lists on Github, with authors who have already been published there able to endorse other papers. I believe the mathematics community is trying that.

The funny thing is that you'd think that arXiv would suffer from the same problems because journals and conferences exist.

People who submit to and review for conferences have known for quite some time how random that process is. Also they are slow. In a field that advances leaps and bounds on a monthly time scale, holding a conference half a year later or publishing a journal article more than a year later is just equal to irrelevance. The reason conferences still matter is because of legacy systems that are slow to adapt to the new reality (and because of networking opportunities).

But actually, thinking back, 10 years ago there may have been more of a stigma around a paper that was only on arxiv.

If your result is not worth publishing in a journal a year or two from now, it't not worth submitting to arXiv today. Just write a blog post about it and move on. And maybe write a paper once you have discovered something of more lasting value.

It's worth publishing somewhere because PhD students need to graduate. And there are many PhD students because it's a ticket to industry labs. And because many professorships around popular topics were created. Those people need to graduate, so there have to be papers. Blog posts don't let you graduate. A good arxiv paper that just got rejected for unfair reasons, but otherwise gets a good community reception can still count towards your graduation if you have a sensible advisor.

Not sure if you understand how the sausage gets made. People won't spend effort at scale (there are always exceptions) on stuff that's not incentivized, such as blog posts. If you work on something, the goal is to make it count. But if you fail on that, arXiv is a second best option.

Also, papers that get published at conferences also get out of date pretty fast anyway. That's just how fast moving fields work.

But things may end up moving the way you are saying, as long as incentives keep up.

Journals (1-2 years) -> conferences (6 months) -> arxiv (weeks to write once the work is done, then days to publish) -> next thing (immediate AI writeup, code, results, tests, open scripts, weights, full reproducibility, full access allowed for other AIs to scrape and learn from, and reorganize and recombine on a speed of days or hours from result to adoption)

It's not been my experience that papers published in CS conferences get outdated quickly. Maybe AI/ML is different, but it's a weird edge case that should not dominate the discussion. Even CS as a whole is an outlier, because most fields that use arXiv don't have conference papers. If the expectations of AI/ML or even the wider CS are too different, maybe they could start a preprint repository of their own.

One of weird aspects of CS research is the tendency to publish small papers. In more experimental fields, a paper may represent years of work for the first author. But CS students often finish multiple projects every year. Maybe it's because of the focus on conferences. You want to attend conferences (to build your networks, and because it's interesting and valuable), but you often don't get reimbursed if you don't have anything to present. Students from other fields can just present their work in progress, but CS students have to come up with publishable results.

Maybe CS should also strive towards bigger projects. Instead of publishing individual results (which someone often improves a year or two later), the papers could focus on the outcomes of entire research directions. That way you are more likely to produce something of lasting value. Something that is worth writing up and rewriting and polishing, until you understand the ideas much better than after the initial write-up.

> Maybe AI/ML is different, but it's a weird edge case that should not dominate the discussion

I wonder about the recent proportions. Growth in ML papers has been mind boggling and I wouldn't be surprised if a major part of Arxiv's submission spike was in AI ML topics.

> Maybe CS should also strive towards bigger projects.

The current rule of thumb is that you need at least three accepted papers in good venues to graduate with a PhD. Until that changes, the incentives point towards trying to publish often. And PIs want this because they also get evaluated in this publish or perish system. This is my main point. Throttling is a stopgap solution because academia doesn't want to admit its internal problems that have been boiling over already before AI.

What makes publishing it valuable? At best it's a second round of PR for the paper, after it was initially released on the arXiv. Reviewers sometimes have good suggestions, but you can get suggestions from people on your arXiv paper and post a v2.

From the perspective of advancing science, journals seem mostly pointless and most of their benefit could be had at next to no cost if we just agreed to publish on the arXiv and pair it with a review system.

From the perspective of advancing scientific careers, because of how founders and institutions treat arXiv papers, you have to still publish in a journal, and so people keep publishing in journals.

But like, at least in physics time passes at the speed of arXiv posts, and getting published usually happens after it's been cited many times already. And if people are already incorporating it into the web of knowledge/responding to it etc., then what did the journal do?

In my experience (in CS, bioinformatics, and genomics), the first round of reviews often improves the paper substantially. You are getting feedback from people outside your usual peer group, and they tend to notice things people closer to you don't.

And writing is thinking. Because you are forced to iterate on the text over a long period of time, your understanding of the topic improves and you learn to express the ideas better. Your job as a scientist is creating new knowledge, and the initial publication of new results is only the first step towards that.

> You could do some kind of karma system for example, where the prestige of your institution, your own karma, number of citations, papers published in high impact-factor journals etc would affect karma weight.

The story that academia likes to tell itself about itself is that it is skeptical and iconclastic and objectively evaluates each piece of science on its own virtues and it doesn't matter who says it. A wrong thing said by someone famous is still wrong, and truth spoken by a nobody is still truth. Of course reality is a bit further from this ideal. But outright admitting it would be hard.

It's a bit like the "in America, anyone can become president!".

>But outright admitting it would be hard.

"Science progresses one funeral at a time" - Max Planck.

If you can't understand Vedic Vortex Structure without such karma system, your understanding will be bad anyway. Even legitimate publications are easy to misinterpret and sensationalize, social dynamics affects it too.

Arxiv is the principal forum for publishing mathematics research of quality. Journals serve a purely archival and bureaucratic purpose; almost no value is added by the refereeing process (probably more is lost) and journal publishing neither improves accessibility nor diffusion of results.

I agree fully that Arxiv is the principle publishing platform for mathematics. However, I have almost always gotten good value out of journal peer-review. It might just be that my sub-field is tightly knit, but I always receive thorough reviews with mostly good questions and suggestions for improvement. And I try to give the same when I do reviews of my own.

Even critical remarks are in the spirit of "You should do this better!", never the kind of gate-keeping bullshit I have seen in other fields.

Moderation for Arxiv is a silly idea. It already shouldn't be taken as a quality signal that something is able to be up on Arxiv. Obviously they should remove illegal stuff, but beyond that, requiring moderation is a misunderstanding of their role and reason for popularity.

Arxiv papers are regularly cited and assumed to be suitable for peer review. So some standards need to be maintained . it's not a free for all.

How about reading before citing? The bar was already very low for getting into Arxiv. If you used that as a quick decider of whether the work is good and should be trusted, you were already doing it wrong.

>If hosting is too expensive, I guess an alternative aggregator could also arise.

google drive

Meh. While I agree with the points about metrics, somewhere you got lost and forgot the bigger picture.

Ref.:

https://news.ycombinator.com/item?id=49952698

Agree that it basically shouldn’t be necessary. But unfortunately Arxiv isn’t able to change the bad incentive structure that the broader hiring ecosystem has set up, right?

If academics and their committees actually deeply considered the scientific qualities of applicants, gaming the numbers would not matter. So I just find it hard to blame AI slop instead of the broken process that academia has settled on.

Part of the reason that the value of Arxiv papers has gone up, is that it's pretty well known that the conference review system is already quite broken and random and lazy and superficial. So being rejected from there is not a good reason to ignore a paper, so conference-rejected Arxiv-only but good papers are a regular occurrence. If conference review had better signal, arxiv could have zero signal.

The other issue is the time delay. Conferences have stopped being an actual place to learn about new work, the way it had been 10+ years ago. Now it's all outdated stuff, and as a specialist you already know the important papers months before the actual conference, so the conference is a networking event basically. The real exchange of ideas moved to Arxiv and Github and social media, with its own problems, hype, algorithmic engagement optimization incentives etc.

But the pressure that shifted away from conferences to arxiv is continuing. The old guard was pearl clutching already by the erosion of conference rubber stamps as this big prestige. The newer ones are now trying to hold back the tide at the Arxiv line. But it will keep on moving faster and faster. And what is going to matter in the end is not how things used to be, but what delivers actual value. If the slop is slop, it will dwindle. If it starts to be actually good, all this will just turn into basically a dock-unions-opposing-automation story.