Robot scientist finds less than one third of scientific results are reproducible | Hacker News Reader

Robot scientist finds less than one third of scientific results are reproducible

cam.ac.uk

39 pointstakiwatanga23 comments

Something doesn't add up:

" ...attempted to reproduce the 74 results. Statistically significant evidence for repeatability was found for 43 papers, meaning that the results were replicable under identical conditions; and significant evidence for reproducibility or robustness was found in 22 papers, meaning the results were replicable by different scientists under similar conditions. In two cases, the automation made serendipitous discoveries.

While only 22 out of 74 papers were found to be reproducible in this experiment,... "

Shouldn't that be 9 were NOT reproducible since 65 we're reproducible?

Or does "results were replicable" mean something other than reproducible?

Probably the 22 is a subset of the 43.

Their sample size was only 74 papers from one field of study.

It's easy to get onto the "science isn't working" bandwagon, but how about some more convincing evidence to prove it's happening as widespread as the claims they're making are.

You know... maybe treating it a bit more scientifically to prove your point.

https://en.wikipedia.org/wiki/Replication_crisis#Prevalence

https://en.wikipedia.org/wiki/Hypergeometric_distribution

If it doesn't work for a well-picked sample, it probably doesn't work for the real world.

things like replication crisis being a matter of discourse is part of the scientific process. scientists notice results don't add up and slowly over time the thing corrects.

But is this paper reproducible and verifiable ?

Balaji Srinivasan recently discussed the importance of scientific reproducibility on The Knowledge Project: https://www.youtube.com/watch?v=NlY8HICFiRs

It is long past time when the majority of paper results could be written in machine verifiable language.

I don't know how you'd even test the accuracy of the "robot" without having a human do it correctly to compare with.

"trust the academia"

Said who?

paraphrase of the director of the national institute of allergy and infectious diseases

https://duckduckgo.com/?q=fauci+%22trust+the+academia%22&t=h...

No results.

https://html.duckduckgo.com/html?q=fauci%20%22trust%20the%20...

erroneously referring to academia as 'science', but it can't be interpreted any other way

Sounds like the error is yours. Trusting science isn't tantamount to trusting academia.

science is a process, not an assertion of the correctness of a certain viewpoint

'trusting science' is a paradoxical phrase, and when used in the context of coercing behavior and thought in a particular direction is anti-science, no matter how peer-reviewed and authority-supported that direction is

And when the process leads you in one direction?

probably a useful model in that direction

I went through the first 8 results or so, and none of them actually provide a quote of someone saying that. Rather, they seem mostly critiques of the mantra, which I assume was being used on Facebook and things at the time most of those articles were written.

There's a universe of difference between avoiding blind faith and distrusting "academia", which is a very broad category. We all only have the information that is available to us.

I doubt even the authors of these papers would tell you to blindly trust them.

They should have used a smarter robot.

Reply on news.ycombinator.com