Hacker News Reader

Comment by underdeserver | Hacker News Reader

It sure sounds promising. Lots of things do. This is why we have peer review and standard publishing practices.

This idea never sat right with me. Only one of Einsteins paper's went through the now standard practice, and he was upset that it did.

Peer review is useful, but let's not pretend it's the only way forward for a theory of everything. It's a very, very modern practice. We did a lot of great physics before it existed.

Nearly everything wolfram does is available to anyone who is interested (including code!).

And just to add on top of this - there is to the best of my knowledge little if any research that shows that the current practice of publication and peer review is actually effective at weeding out bad science. The current replication crisis in psychology and other fields would suggest that it's actually quite bad at it. For a community that is supposed to be based on the ideal of experimentation and falsifiability, scientists are surprisingly willing to just accept at face value, that the current process is sound, without actually scrutinizing this hypothesis.

> The current replication crisis in psychology and other fields would suggest that it's actually quite bad at it.

Not saying that you're necessarily wrong but be careful not to compare apples to oranges: There is a huge difference between fields like math & physics on the one hand and fields like psychology on the other.

Definitely, but both use peer review.

But peer review works completely differently in math & physics than in psychology. In math and (theoretical) physics, practically all results (claims) can in principle be reproduced (tested) by other people. Sure reviewers are usually short on time and won't verify every single detail but at the same time – being experts in their fields – they tend to be extremely good at distinguishing the "easy" arguments from more intricate ones and can usually come up with rough arguments as to why (or why not) a given new result makes sense in light of existing results & literature.

I don't think it is accurate to use problems in the field of psychology to indict peer review in math or physics.

The reviewer was Tolman (or Robertson I think Tolman), and it was arguably one of his most important ones.

It's not about peer review, it's about making predictions. Theories that make predictions that come out true are good ("correct"). Peer review is a poor substitute.

Peer review is just a heuristic people use to spend time more efficiently: You know that if a paper is peer reviewed, it's more likely to be worthwhile spending time actively engaging with it than if it weren't.

But that's only statistical - if you want to quickly screen thousands of research papers and the only signal you have is peer review and maybe citation graph - and doesn't apply to specific papers or the research program of a specific person.

Reply on news.ycombinator.com