Hacker News Reader

Comment by gordaco | Hacker News Reader

Wow, that's sad. Are we really so blind when it comes to cancer research?

I recall that oncology journals usually have a ludicrously high impact index (as ludicrously as 5 digit IIRC); that means there are a lot of citations, which is an indicator that there is a lot of research going on. And with a lot of research going on, well, you can expect a lot of false positives. So, I'm wondering, could this be a case of cherry-picking or some kind of selection bias? It wouldn't be difficult to select a lot of bogus-sounding research and test it.

View on HN

Medical research in general is dubious at best. See for example http://www.plosmedicine.org/article/info:doi/10.1371/journal...

There's even a startup trying to work around it: http://www.metamed.com/. They look for papers related for a given condition on request, trying to find ones that are actually well done and promising.

"And with a lot of research going on, well, you can expect a lot of false positives."

Why would you expect a lot of false positives? Aren't results supposed to be reproducible at least 95% of the time?

I was thinking about absolute numbers, so 5% of a lot can still be a lot.

But who is going to pay to reproduce that research? What if that research took years to do?

Why do the research in the first place if the only outcome is a number, floating in space, disconnected from anything and unreproducible?

You're still sort of operating on the idea that the unreproducible papers have some sort of abstract value to them, and therefore we shouldn't slow the flow of them lest we ruin their value. But they don't. They're worthless. They're worse than worthless. They're of negative value. It would be far better to slow down and verify that what we think we know is actually true, because in the end that would actually both faster and a more efficient use of resources. Basically, instead of learning worse than nothing (thinking we know something but actually being wrong), we'd learn something. That's a pretty decent upgrade.

You don't have to do the entire experiment, you can just do a proof of principle and then build on it. For example, a lot of studies in science have 1 big idea, and they show it works using a variety of different methods. Pick 1 method, show consistencies, and then build on it.

One of the serious problems in science is that people state what you said above and will jump in feet first on a huge multi thousand $ study, but never spend the time to validate the model in their own lab. So you waste that money initially, then go back to figure out what went wrong. $$$$ being wasted. (The lack of negative data publications is also a massive problem that contributes to this)

What I'd like to see (and I'm currently working on) is raw data from labs. I do xyz experiment, publish it, and release all the raw images, excel files, raw data outputs etc. Now other scientists can go into those data, ask their questions and try to reproduce it or at lease provide a different perspective. This is what I'm working on at http://omnisci.org and I intend to bring to the scientific community. We need transparency, because this "behind the door" shit (peer review, grant reviews, only positive data etc) isn't working.

Reply on news.ycombinator.com