* Examining the presentation of tumor-associated antigens on peptide-pulsed T2 cells (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3894244/)
And if you want to study the effectiveness of any immunotherapy, it's very common to implant a mouse cell line like B16 in live mice and then see if the tumor regresses under different treatment conditions (e.g. with or without checkpoint blockade).
Example:
* Localized Oncolytic Virotherapy Overcomes Systemic Tumor Resistance to Immune Checkpoint Blockade Immunotherapy (http://stm.sciencemag.org/content/6/226/226ra32.short)
...but really every preclinical paper on a new kind of vaccine or immunomodulatory antibody uses cancer cell lines in this way.
In some ways I think these kinds of things are more about having the time and resources to keep things organized. It is a constant struggle in labs by people who may be expert biologists, but amateur database managers.
If anyone who points out errors is being pedantic, it makes you wonder about the point of doing all that. Just come up with an idea A, say "heads = conclusion A", "tails = conclusion not A", and flip a coin.
In graduate school, as part of a class on survival analysis, we subjected the same data set to increasingly sophisticated analysis techniques to account for all kinds of things.
Each time, the effect estimate changed.
At the end, while discussing them, the professor asked a very simple question: "Do any of these suggest that HAART is a bad idea?"
Similarly, during the Ebola epidemic, when everyone was fussing about the various forecasts missing the mark, etc., what they were actually predicting was "This is a serious crisis that needs international intervention".
I'm also sure that there are many things going on at any given moment that should count as "serious crisis that needs international intervention". The real question is whether it is a more serious crisis than other things currently happening.
To deal with both these issues you are going to need deeper understanding than "good idea vs bad idea" or "is crisis vs not crisis".
But the real problem with being unable to quantify your understanding is that you are left unable to make precise predictions, thus you can never perform any stringent tests. If all you can predict is something vague (eg, HAART will increase 5 year survival), there will be many ways to misinterpret the data in support of your explanation even if it is totally wrong.
> To deal with both these issues you are going to need deeper understanding than "good idea vs bad idea" or "is crisis vs not crisis".
This is not, actually, how the response to Ebola worked. It was very much "crisis vs. not crisis". And a huge amount of clinical decision making is "good idea vs. bad idea".
The suggestion was not that you're not able to quantify your understanding. The suggestion was there are errors that are possible to make that, while changing your effect estimate, do not change whether or not you do a thing.
To return to the HAART example, imagine you're an HIV+ patient and I've told you that a major study failed to control for time-varying confounding, and that upon re-analysis, instead of doubling your 5 year survival, it only increases it by 87%.
Or, for a form of "is this repeatable?" that I particularly despise, that it still doubles your chances of survival, but the p-value has gone from 0.047 to 0.062.
Do you want to stop taking the drug?
Or, for a form of "is this repeatable?" that I particularly despise, that it still doubles your chances of survival, but the p-value has gone from 0.047 to 0.062.
Do you want to stop taking the drug?"
This obviously depends on the relative costs. Cost of side effects, buying the drugs, time going to treatment, etc. That all requires accurate quantification. But cost-benefit isn't even what I meant.
To begin with, if they can't nail down a quantifiable effect that is stable from study to study, who knows what is going on? Why would you have confidence in their estimates of effectiveness if they are inconsistent with one another?
I'm going to suggest if you're facing death from an AIDS-related illness, a relative risk of 2.00 vs. 1.87 will feel very, very similar to you.
"To begin with, if they can't nail down a quantifiable effect that is stable from study to study, who knows what is going on? Why would you have confidence in their estimates of effectiveness if they are inconsistent with one another?"
Define "stable" - because even if the effect of something is fixed in the same way a physical constant it, it's invariable sampled with error.
Again, if I give you five studies that suggest that HAART improves survival by:
100%, 102%, 87%, 94% and 96%
are you really going to suggest that we don't know that HAART improves survival?
I at first thought we both understood there would be some uncertainty about these values no matter what but we were leaving that out out for simplicity's sake. Roughly, I was assuming it is at least +/- 10-20%. The reason these values "feel very, very similar" is that I would not expect medical data to be able to distinguish between them.
>"Define "stable" - because even if the effect of something is fixed in the same way a physical constant it, it's invariable sampled with error."
Ok... so you are implicitly considering uncertainty.
>"Again, if I give you five studies that suggest that HAART improves survival by:
100%, 102%, 87%, 94% and 96%
are you really going to suggest that we don't know that HAART improves survival?"
Not enough info.
- Where did the uncertainty go?
- What methods were used to generate these values? Even basic strategies like blinding the people collecting/processing/analyzing data is often still missing.
- What population/frame do those numbers refer to, and how much error can we expect from extrapolating for other situations in the future?
You need a reliable quantification, doing this hand wavy "things are better/worse, significant/insignificant" is an awful idea.