Comment by piva00 | Hacker News Reader

Comment by piva00 | Hacker News Reader

Parent Full thread

The problem is even worse given that for another researcher to challenge fake data they are also putting their reputation and career in jeopardy.

I've seen this happen with a friend of mine during his PhD, working in a climate science department doing simulations of cloud formations at the molecular level. Some 2-3 years into his research he kept bumping his head against a model that wasn't producing the data the research team expected, he spent almost a year trying to contact the author of the paper where said model was introduced, wanting to get all the dataset and initial conditions the author used so he could replicate it. The author is a quite prominent professor in the field, working at a major British university and simply wouldn't provide it to the research team, datasets had missing pieces, he deflected multiple asks for the initial parameters of the model, etc.

In the end my friend uncovered that it was most likely faked data and to prove that was another year of a battle between heads of departments which led him to a complete burnout and to lose trust in science. It was such a hardship to endure, I saw him mentally and physically suffering, and questioning said professor reputation was way above my friend's paygrade and research efforts, it was a political nightmare and after he finished the PhD (as fast as possible without exploring all of his research's goals) he just dropped the case completely.

Academic research seems to be in such a mess, perverse incentives all around and very few are willing or even able to question bad science.

Thats my point. Peer correction doesn't work with someone cheating, as it can do with in normal mode scientific process. If there was a solid legal framework to deal with this your friend could have passed what he found on to the police. Institutions 'investigating' their own researchers is a recipe for disaster, but the surrounding society has levers to control the process. Literal fake scientific work shouldn't be dealt with like it is bad science. It shouldn't be an issue of politics but responsibility and punishment.

Just scanning the wikipedia article on scientific misconduct, https://en.wikipedia.org/wiki/List_of_scientific_misconduct_... , it looks like Biology and Medicine are the most likely to suffer from fabricated evidence.

Physics, aside from the major scandals of Ninov and Schon seem to be much cleaner.

Then there is the replication crisis and sokal and its spawn repeatedly showing a lack of rigor and scrutiny in certain fields.

People act shocked when there the public distrusts "science" so much. I don't think its surprising at all. People don't distrust science. They distrust scientists. And there is ample evidence to show they are just as flawed and fallible as anyone else

Open code and data. It's de rigueur in a lot of the more prominent social science journals for obvious reasons.

I will say, however, that we need to be very clear about making a distinction between bad methods and intentional malfeasance. Academic research generally involved a small team of researchers, at best. The code and amount of overall effort involved for what the earlier poster described is extremely intense. For better or worse, code QC simply doesn't exist in most departments for largely financial reasons.

You can generally expect that - most - efforts fall short somewhere, simply because the odds of not making a mistake or two in a 15K+ code base with one or two non-professional programmers are almost nil.

The hope is that another team in the future will tackle a similar task and, in the process, obtain results that are or are not consistent with what were previously published. Thankfully, if the question is interesting or impactful enough, and climate research certainly is, the aggregate body of evidence will eventually force out unintentionally bad research.

Of course, open data and code requirements facilitate this.

Reply on news.ycombinator.com