I don't quite get this point: this is cancer research, co in theory all related institutions should be interested in replicating breakthrough studies, right? And failure to replicate it, time after time, should be a cause of concern, shouldn't it?
The lifecycle for a junior PI is about 5 years, which really means they have about 3 years to produce a result sufficient to advance. This is enough for a bit of validation but unlikely anything beyond their lab, so the result is likely to be untested. But who in that lab cares? They just need to get the next job, so a 3-4 year timescale is fine. The PI cares long-term because a lack of new productivity means a lack of ongoing funding, which means they either improve the quality of their processes (which means herding the junior researchers through that system) or simply continuing to hack their way through.
There is a reason why people moan about scientific theories advancing one death of an old professor at a time.
For external industry funders, they have a strong and vested interest in identifying good output before it is public. The strategy here typically is a mixture of identifying good researchers and making them kings (consistent funding, direct lines of communication, access to nonpublic materials, etc), and of placing lots of small investments to get in the room for discussion. An example of this mixture is the BP deal at UC Berkeley [2].
As for shifting standard of care, this is a commonly-expressed goal but often the research is far enough away that the individuals doing the work lack close communication with the actual practitioners. Some institutes are trying to resolve this by close physical and social proximity (e.g. TU Dresden's medical campus) but this is far from the norm.
[1] https://www.helmholtz-muenchen.de/fileadmin/Jahresbericht/Ja... [2] https://www.berkeley.edu/news/media/releases/2007/02/01_ebi....
But this is the opposite of how it should be! If we learned anything from scientific research is that if a breakthrough study is published, it means almost nothing until it gets verified. Actually, these days I tend to believe meta-analyses almost exclusively. But you need time for these to appear.
So when I read about some exciting new study, my first reaction is, "Well, that's interesting, we'll see what other scientists say." So actually we should look forward to the first replication as the first indicator of whether the original study was correct or not. The authors of the original study and the journal board should be grateful to the replication team for helping them do their job! Rejecting replication is a disservice to science.
Well, no. Grants don't pay for replication. Replication studies are less likely to get published or get citations. In other words, if you want to do replication, you wont be able to find money for it. If you find money on it, there will be punishment in terms of lesser career progress.
The system does not value replication.
Well, most of us (scientists in general) do care, but we have opposite incentives.
I share your concern. The operative word here is “should”. I. Practice replication studies have an opportunity cost because they won’t end up in a prestigious journal, they are harder to use in grant applications, and the time and resources could be spent on studies that would be more useful in these ways.
> And failure to replicate it, time after time, should be a cause of concern, shouldn't it?
It would be, and it would also indicate that a technique is dodgy, or that a research group or scientist is unreliable.
Not necessarily: it could also mean that we don't properly understand the process, and therefore essential steps/prerequisites in the experiment are left undocumented. That type of error is more fundamental than merely the technique or the researcher.