Arxiv.org reaches a milestone and a reckoning

https://www.youtube.com/watch?v=gv5mI6ClPGc

(Discussion w Noam Chomsky)

That's overly optimistic. What you mean is "with peer-review you know that someone will at least skim your work and maybe catch errors."

I agree with the rest of your points though. Zero peer review is probably not a great solution.

I think sometimes reviewers hide behind their anonymity to give snarky responses, and using their power over you, knowing you will try your very best to implement all of their suggestions in order to publish faster. I just think the whole process should be more transparent.

arxiv was about distribution. It didn't replace peer review - articles were still submitted to journals and published there too.

If an article was posted to arxiv and not a journal, the odds of a citation went down massively. And the journal it was submitted to was a factor in whether or not we read it. When articles were eventually published, most authors also updated the preprint with the post peer review version.

Basically it meant that (1) it was easy to keep up to date with what everyone was working on, and pick up interesting new stuff (2) most citations, post 80s, you saw in whatever paper you were reading, you could look up on arxiv and be reading it in seconds.

Or, instead, you get a much larger audience reviewing your article, instead of the 2 to 4 reviewers involved in the paper publishing process :-)

You can get comments from critical and interested readers, you upload a new version of the paper to arXiv, and repeat the process until the article is ready for submission/publishing.

In other words: I think that an article on arXiv (on average and as a whole, individual articles may be exceptional) is subject to much wider and more extensive scrutiny and verification than the average article during the paper publication process.

I wonder if academic papers could benefit from a similar process of communal development (though it flies in the face of the dead-tree format of academic papers, but we're talking about Arxiv here, not Elsevier). Even if suggestions don't get merged, having a repository that captures discussion around the central artifact is also useful (such as when a maintainer decides that a feature doesn't belong in a project).

Peer reviewing is essential, of course. But note that in most journals reviewers are not getting paid. They have even less incentives to find mistakes in your work than you are. I have no idea how such an utterly broken system emerged, where journals siphon billions of tax money exploiting the work of scientists.

This is a remarkably naive (and charmingly optimistic) view of peer-review.

For the vast majority of the history of academic research peer review did not play a major part. It wasn't until the 1970s that the modern day peer review system emerged and it did so as a reaction to a funding crisis in academia to produce the illusion of legitimacy.

It of course has done no such thing. It has done nothing to limit the reproducibility crisis which has shown it's ugly head in nearly all scientific fields of study. One could even argue that peer review coupled with a publish or perish culture is the cause of such a failure of science.

This also means just about any great scientific discovery you can think of prior to 1970 did not go through the peer review process as we know it today. If science seems less exciting today, peer review is certainly one of the reasons for this. I'll leave you with Geoffrey Hinton's words on the subject:

> Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad.

Laking peering review is a feature, and not just because it speeds up time to results.

The situation for conferences in CS is basically how it works in journals in other fields. There may be more low-quality submissions to journals, but at the same time, the extreme competitiveness means most high-quality submissions are rejected also. The "solution" has been the appearance of open-access journals which are less concerned with subjective reasons for rejection like "impact", and scale to accept more papers. But they cost thousands of dollars. Open archives are a more democratic solution.

We still don't have a proper open /free tax filing too from the IRS or tax computed by IRS only for us to verify/confirm as in some of other countries, primarily because intuit (and others) lobbied to make sure the government would never develop one (encoded in law no less), so it is not that surprising that a community benefiting project that threatens some big business revenues has no government funding available.

If you're suggesting it as alternative for academic publishing in general, it would cost a lot more.

Arxiv isn't a "publication platform" in the sense that peer-reviewed journals are. The cost of hosting and distributing PDFs is close to zero, obviously. What does cost money is the copyediting and running the peer review process.

People think Elsevier are the worst people in the world (not entirely wrong) and taking in millions while exploiting the free labor of authors and reviewers. But it's relatively easy to quantify the value they do add: their financials are public and, last I checked, they have operating margins of about 30 %.

Let's say we could do without their marketing and billing departments and that gets us to a margin of 50 %. That's pretty good, but it isn't quite the rip-off people sometimes suggest it is. Replacing the commercial publishers with government-funded open-access journals would be a billion-dollar endeavor, not a few million for servers and a bit of software.

On the other hand, PubMed exists.

My guess is, for a preprint archive, network effects are paramount, and ArXiv was there first. It’s not directly a government project, but is still funded by US government grants and hosted by Stanford (now that the international mirror network is shut down, which does give me a slightly jittery feeling).

[1] https://hal.archives-ouvertes.fr/

[2] https://groups.google.com/g/isl-development/c/JGaMo2VUu_8

Here in the UK, there is a big research evaluation exercise called the REF. The REF is important because it largely determines how the government allocates funds. In order to be eligible to be evaluated, researchers have to essentially submit their pre-prints into open access.

What this has meant for the last 10+ years is that most universities participating in the REF have an e-prints server, like the arxiv. Obviously, being publically funded, universities are then using government money to pay for these costs.

Now your question might be whether it's sensible for the entire country (or research councils) to operate their own e-prints server. I don't know the answer to this.

All I know is that the more bureaucracy you involve, the more bullshit results.

So on one end you could takes something like Newtons Principa, that synthesised a lot of the known facts and laws at the time into one single coherent framework, and on the other you can have Galileo recognising that he was seeing surface features on the moon and not just patterns on a flat disk and announcing this will some illustrations showing the principle and not what he actually saw.

As to the first part of the question, if you talk of "publishing" in academia you usually mean you want your work to be know by other credible researchers in the hope they find it useful.

The most reliable way to achieve that without credentials is to befriend someone respected in the field you are interested in before you send them your results, because they are more likely to have patience with any errors or confusion in your presentation that way.

There are of course some places that pride themselves on being completely free from filtering, but the concept of "spam folder" along should tell you how good a chance you have of getting any serious attention in those places.

A more approachable avenue could be blogposts and videos. One good example of a scientific blogpost I recall is here: ( https://dynomight.net/2020/12/15/some-real-data-on-a-DIY-box... ) Someone tested very simple air purifiers with limited experiments. It follows the same "abstract-methods-results-discussion-conclusion" format you might expect. Some obvious caveats, this data could have all been faked, and it isn't peer reviewed for methodological flaws. But you can reproduce the experiments yourself for ~$150.

I am in academia but I like blogposts a lot more. I wish they were more popular. I think HTML makes more sense than PDF, and to some degree they cut the middleman out between communicating to peers and communicating to other academics.

HN is a nice tidy list of top things, is there an equivalent for ArXiv?