New treatment destroys head and neck cancer tumours in trial
theguardian.com
theguardian.com
Note that the actual trial referenced by this article, CheckMate-651, failed its primary endpoint [1]. The positive result is only for a subpopulation of patients with high expression of one of the targeted proteins.
Anyway, if you haven't heard about immunooncology, it really is an amazing field with lots of great results. But this particular article about very incremental work.
[1]: https://www.onclive.com/view/nivolumab-ipilimumab-misses-os-...
He's still alive 15 years later, despite the survivability for Melanoma that has spread is dire. That's, of course, a single piece of anecdata, but the power of biologicals in how we treat disease going forward cannot be understated.
https://www.icr.ac.uk/news-archive/immunotherapy-combination...
And includes this paragraph, that I just can't understand:
"The results were not statistically significant, but the immunotherapy combination, designed to spark the immune system into action against cancer, led to a positive trend in survival when compared to the ‘Extreme’ standard in a group of patients - those with tumours that had high levels of an immune marker called PD-L1"
How is this "not statistically significant" but at the same time "...survival rates... were the highest ever reported..." ? I don't get it.
You can't draw conclussions from subsets of the study. This is a rule that even professionals often forget.
Perhaps "we did something, and while we're not certain whether it was what we did or luck that made the difference, it was hugely successful and deserves more research."
In scenario 1, you have a coin that's weighted towards head with a probability of 75%. You want to confirm that it really is an unfair coin, so you flip it 10 times and get 7 heads. In casual conversation, you or I might say that 75% is significantly more than 50% you get with a normal coin. However the result of your experiment isn't statistically significant; a fair coin could have easily given the same results by chance.
In scenario 2, you have an unfair coin that comes up heads with probability 50.1%. You flip it one billion times, and 501 million of those are heads. Now we have the opposite situation: the difference between 50% and 50.1% is insignificant in most contexts. But the results of this experiment are unambiguous; over 1 billion flips, it would be extremely unlikely for a fair coin to show that large of a skew. So the results of our experiment are statistically significant and we can conclude that the coin really is unfair.
Now we need a second explanation: If my possible explanation is correct, why both groups lived more than usual?
Perhaps a similar study started a short time before them and siphoned the worst case so they get only patients that were slightly better than usual?
Perhaps their definition of "terminally-ill" is less strict than in other groups that make a similar treatment and their patients were slightly better than usual?
Perhaps someone added a vitamin as a standard baseline treatment two years ago, and now everyone gets a few more months of survival?
Perhaps during the pandemic the family had more time to take care of them and that increased the visits to the hospital and that increased the survival time?
I can continue making up excuses, but it's very difficult to know without reading the study, all the similar studies and making a very deep research.
The important conclusion is that without a control group, it's very difficult to be sure what is the baseline and if the new drug (combination) made a difference. Preferably a doble blind randomized control group, not an unrelated bunch of guys in another city.
A "statistically significant" result is a result unlikely to be explained solely by chance or random factors. It has a very low chance of occurring if there were no true effect in a research study. It does not coloquially mean it's wrong, or unimportant.
The p value, or probability value, tells you the statistical significance of a finding for a given measurement confidence interval. Than can happen for a variety of reasons.
Consequently, a result is significant or not because of all the parameters of the study, for example measurement characteristics, number of samples, etc, but not its results.
While not wrong in the absolute, your answer mixed in examples of unknown cofounding variables which could be the cause of the specific study's result, but they are not in the strict sense examples of statistical insignificance, nor examples of causes of it. Their absence wouldn't have made the study any more significant.
The study OP referenced had 947 patients over ~50 mo. [1], which ought to be a sufficient sample size, but those were split into two groups with different regimens [2]. A third and half had to be excluded because of how measurement was made or after review of the cases [3]. With ~320 per-group, the sample size is not that great anymore, so result might not be that precise, depending on the chosen confidence interval.
Indeed, drilling down [4], we see a 97.5 CI, and probability values for each group of 0.49 and 0.046. That doesn't mean the observed trend is wrong, just that we can't know for sure - hence it's "insignificance" in statistical parlance.
The absence of those effects given by in your examples wouldn't make it any more significant. What could make it more significant ? For example more patients, less exclusions, better confidance interval. If we look at the discussion in [3], the researchers themselves believe a problem was the measurement itself : "A limitation of this analysis is the relatively short duration of follow-up. As of the data cutoff date, the median overall survival was not reached in either group; follow-up is ongoing."
A note on double blind studies : in this particular case, a double-blind randomized control group study is not practicable, as it would be a gigantic breach of ethics.
DBRCT is not an acceptable research technique for all medical scientific endeavors. It has its applications, but this wouldn't be it. You can only measure difference with other cures if such cures exist, not with the absence thereof (i.e. a control, placebo group) if they do exist. A DBRCT'd mean you'd have to refuse care, which is not possible.
Moreover, the medicine was experimental, you can't double blind, as the doctors in charge of the patient must be able to react quickly, and those might not be the clinicians in charge of the study.
Consequently, the study's author were right in making it a simple CRT.
Don't confuse methodology with medicine.
[1] https://clinicaltrials.gov/ct2/show/NCT02741570
[2] https://www.thelancet.com/journals/lanonc/article/PIIS1470-2...
[3] https://www.nejm.org/doi/full/10.1056/NEJMoa2026982
[4] https://www.cancertherapyadvisor.com/home/news/conference-co...
This is true, but sadly in most reports it jst mean p>0.05 in spite they should use a smaller value due to the look elsewhere effect and other stuff.
> A DBRCT'd mean you'd have to refuse care, which is not possible.
No, double blind means that both arms must receive the same visible treatment. It can be a new drug vs sugar pills, or a new drug vs an old drug. If there is some standard treatment, in most cases the control group receives the it.
> Moreover, the medicine was experimental, you can't double blind, as the doctors in charge of the patient must be able to react quickly, and those might not be the clinicians in charge of the study.
What type of reaction is possible if the doctors know in which group each person is?
I agree that in some cases a double blind experiment is impossible or very difficult, but looking at many of the proposed miracle drugs against covid-19, one of the patterns is that most studies have a "control group" that is not a real control group.
However, if the treatment only went to 5 patients, it's possible that you just happened to get 5 straight patients who would have survived anyway. The probability of that is pretty low (~3%) but you'd need more patients to conclusively prove it.
So I think they're saying this was a better result than the standard care, but there is more than 5% probability that was by luck.
(Warning: following numbers are made up for the sake of argument - IDK how it really looked like in the trial.)
If the drug does nothing to, say, 95 per cent of patients but completely cures the remainin 5 per cent (those who have the "right tumor genome"), taken together the effect may be too small to reach the threshold of statistical significance. But for those five percent, it is a vast difference.
There is basis for that, I can't conjure a specific Oncology textbook right now. Short explanation: the process that led to the production of cancer cells is still ongoing and will lead to cancer cells again given enough time.
You are contradicting yourself by saying that. I would say it is pointless and unprofessional.
You could say "Although the results don't support the hypothesis/approach/treatment, we will keep trying because we want this to work badly"