Hacker News Reader

Comment by 0cf8612b2e1e | Hacker News Reader

Those are two wildly different things. Most things are flawed. Deliberately fabricating results is anathema to advancing science.

View on HN

Yes I work with these IPD spreadsheets every day and they all have typos often in the time/date column (issues with midnight and new years are very common) this can result in an erroneous reporting of something like average drug exposure or clearance if 1 or 2 subjects have uncaught errors in the reported data

I would expect 26% or more studies have these flaws but faked data is a different thing entirely and 26% fake would be incredibly worrying

How many studies are faked and how many just have a 'selective outcome'? Almost every nutritional study I read is at some point sponsored by a group that would benefit from a positive (or negative) outcome. I imagine several studies are run and only the one where the conditions led to a desired outcome are actually published. The researchers may know the results are inconsistent and there is an error somewhere, but it's not in their best interest to find the error and correct it.

That’s why preregistration is important.

True that, but the article does distinguish between "flawed" (44%) and "problems that were so widespread that the trial was impossible to trust" (26%).

Really, though, how/why would we expect otherwise? There's nothing in the system to prevent it, and plenty to incentivize it. There's really no good reason to expect (under the current system) that it would not happen (a lot). Without systemic change, it will not get better.

How do you discern between a study that is flawed and one that is faked? For instance, you could use some methodology that you know to be flawed, and allow that flawed methodology to bias your result in a pre-determined direction. If you're ever caught, you just claim it was an honest error, but it functions exactly the same as if you generated the data wholesale.

He groups them together because ultimately the result is that the science can't be trusted. He doesn't go so far to claim that one was intentionally faked vs gross incompetence.

My interpretation is that "flawed" study usually have honest data but wrong interpretation/analysis or wrong data gathering design so to make the conclusion essentially worthless. "Faked" study just fucking lie and may just provide rigged data to support a flawless conclusion.

They mean flawed to the point of being useless in a way that indicates incompetence or negligence or fraud. Not flawed like they had a typo or a negative result or didn't use MLA formatting in their citation.

Reply on news.ycombinator.com