Hacker News Reader

Comment by Eridrus | Hacker News Reader

I work with a fair amount of people from academia, a lot of them seem to have the view that things that have not been peer reviewed are not worth reading.

But ML already has conferences and journals where work is peer reviewed and published in an open access manner.

So it's not that academics see journals/conferences as value-less, but that they are already doing all this work for free, why do it for free for Springer who will then charge everyone?

View on HN

I think a lot non-academics (myself included) tend to hear the phrase "peer reviewed" and conflate it (rightly or wrongly) with "these results are replicable and therefore correct."

I don't know what the long term solution is but I have a feeling it won't involve companies that charge a lot of money for a service (publishing) that has been effectively free since the Internet has existed.

A lot of peer reviews in ML are currently completely open. For instance you can view the reviews for papers in previous and upcoming conferences here [0].

Also in terms of replicability ML is in a unique position where a lot of research is on standard datasets with models that don't take an excessive amount of compute power to train. This means that if the authors include a github link to their code you can fully replicate their results. So some "peer reviewed" ML papers can actually subscribe to your first description of peer reviewed.

[0]: https://openreview.net/

I think a lot of ML research, despite being replicable is not necessary useful, because what we really want, approaches we can add in addition to all the other methods we have, that introduce meaningful improvements without unnecessary complexity and work across datasets, are quite rare. And that's assuming the evaluation was done well.

Not to say the research is worthless, just that straight replicability is not necessarily enough.

The greatest value of journals isn't that three random people (give or take) have reviewed and suggested edits to the papers before publication.

The greatest value of journals is when hundreds or thousands (or, hell, dozens) of other people, well informed and competent in the subject matter, read the papers and consider how it meshes with their own experience and understanding and build upon it, writing letters to the journal pointing out problems and articles which follow up on the original work.

That is how an idea is validated. An article published in a journal that no one reads isn't much different than an unpublished article, and if work is spread, responded to, and built upon outside of journals, it isn't much different from a journal-published article.

That's unfortunate because that is how consensus and dogmatic group-think emerge. People need to think for themselves.

I love learning but stayed away from college because I couldn't stand how fundamentally at odds traditional education routes are to creativity and free thinking.

Personally, I find Twitter and arxiv-sanity good sources for keeping up with ML. They have their own mechanism of surfacing popular content and filtering based on preferences.

> a lot of them seem to have the view that things that have not been peer reviewed are not worth reading

Can't we have some kind of PageRank system, not necessarily based only on citations, to solve that?

Pagerank is a kind of citations system.

Yes, but I suppose you can also use it for upvotes instead of direct citations.

Reply on news.ycombinator.com