Hacker News Reader

Comment by Macross8299 | Hacker News Reader

>Do they get it wrong? Yes, occasionally. Not frequently. Occasionally. But that also, is science. The FDA follows rigorous statistical procedures, but even p<0.05 (or <0.001) still means false positives can happen.

The issue is that there is a huge lack of transparency. When a third of the studies involved in approval of anti-depressants were withheld from the public [0] and even some withheld at the insistence of the FDA [1], what good do low P-values do?

Hard to trust the science when you're not even allowed to review the papers for yourself.

[0]: https://www.nytimes.com/2008/01/17/health/17depress.html

[1]: http://www.washingtonpost.com/wp-dyn/articles/A9802-2004Sep9...

View on HN

Well, current medical research is not really science, more of a sort of office politics with added prestige and financial incentive.

Actually you can read all the papers you want, you have no guarantee that what's on paper is what happened. Academic and industrial misconduct and epidemiological ignorance are so prevalent in the field that it's more akin to a religious cult.

No, unfortunately I'm not being pessimistic. This is also the case for the FDA people. They can't know, even by doing good work and to a very large extent, whether something will prove harmful or useful.

>you have no guarantee that what's on paper is what happened....

Can't speak for others, but we were deathly afraid of submitting anything dependent on papers that we had not verified ourselves to 100% satisfaction. This actually held us up a lot, but it was safer than taking a chance that you would get it by those people at the FDA. Now of course, we were making software, and when you deal with computer science research, there are a lot of BS papers out there. I remember reading through papers on ray tracing, which had supposedly been peer reviewed, but were presenting arithmetic that was entirely incorrect. Literally every paper we found on this particular algorithm was riddled with almost identical mathematical errors. It was crazy. (I would say that computer scientists have got to be the worst mathematicians I've seen during my time in the sciences.) In any case. I knew that one of the people we were dealing with was a pretty good math guy. Yeah, we decided not to chance it.

Having bogus research out there is one thing, getting lucky enough to slip it by the FDA is something entirely different. Don't mistake those two things for the same issue, it could cost your company. Or worse, if you do manage to slip it by the FDA, it could cost your freedom when your bass ackwards ortho normal basis calculation switches left and right, and the surgeon goes in after the wrong kidney.

During my MSc, I was implementing, validating, and comparing multiple numerical methods for a certain field of Aeroacoustics. Most (peer-reviewed!) papers had errors, some glaring, some subtle. The cynic in me thought those were done on purpose so that results were harder to reproduce.

There was one specific signal change (a + to a - in an equation) that took me almost a month of hitting my head against a wall to figure out...

Yeah, people need to learn that just because it's published doesn't mean it correct. The problem worsens when people just link papers they don't understand when arguing online.

Reply on news.ycombinator.com