Medical-evidence giant Cochrane battles funding cuts and closures
nature.com
nature.com
Hopefully this results in some innovation at the conceptual and financial levels. For those who lost their income, they have transferrable skills that would be welcomed in healthtech companies, other governmental groups, and academic positions. It's really not as bad as it seems.
Criticism is good, expecting perfection is not.
Quality issues are a problem of the underlying research rather than the review process. High quality reviews, like Cochrane, assess risk of bias which can limit the strength of conclusion they make where the underlying studies are poor quality.
> There are newer forms of synthesis like realist reviews, meta-modeling, and model-driven meta-analyses (those are just a few examples).
All of these answer different questions/have different use cases than a SR and are much harder to do while maintaining comparable quality and risk of bias.
What's your gripe with high quality SRs following standards like PRISMA, STARD and QUADAS-2?
I have nothing to offer other than thinking that this is a super cool area of study and I’d love to hear more
My whacky opinion: human facing sciences (medicine, psychology, diet, sociology, economics, etc.) have a huge problem with solutions that require motivation to succeed. In other words, if you are motivated, you can make masks work. If you are motivated, you can make intermittent fasting work. Same for EVs, Solar Panels, etc.
Their findings:
On medical/surgical vs. nothing:
"Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)"
On N95/P2 respirators vs. medical/surgical:
"The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence)."
[0] https://www.cochranelibrary.com/web/cochrane/content?templat...
https://www.cochrane.org/news/statement-physical-interventio...
The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses.
The problem was how they communicated. The data is the data.
"Adapt or die: how the pandemic made the shift from EBM to EBM+ more urgent"
https://ebm.bmj.com/content/27/5/253
"Evidence-based medicine (EBM’s) traditional methods, especially randomised controlled trials (RCTs) and meta-analyses, along with risk-of-bias tools and checklists, have contributed significantly to the science of COVID-19. But these methods and tools were designed primarily to answer simple, focused questions in a stable context where yesterday’s research can be mapped more or less unproblematically onto today’s clinical and policy questions. They have significant limitations when extended to complex questions about a novel pathogen causing chaos across multiple sectors in a fast-changing global context. Non-pharmaceutical interventions which combine material artefacts, human behaviour, organisational directives, occupational health and safety, and the built environment are a case in point: EBM’s experimental, intervention-focused, checklist-driven, effect-size-oriented and deductive approach has sometimes confused rather than informed debate. While RCTs are important, exclusion of other study designs and evidence sources has been particularly problematic in a context where rapid decision making is needed in order to save lives and protect health. It is time to bring in a wider range of evidence and a more pluralist approach to defining what counts as ‘high-quality’ evidence. We introduce some conceptual tools and quality frameworks from various fields involving what is known as mechanistic research, including complexity science, engineering and the social sciences. We propose that the tools and frameworks of mechanistic evidence, sometimes known as ‘EBM+’ when combined with traditional EBM, might be used to develop and evaluate the interdisciplinary evidence base needed to take us out of this protracted pandemic. Further articles in this series will apply pluralistic methods to specific research questions."
Too cynical??
Meta-analysis and systematic reviews were done before Cochrane, and will be done after it. And most pharma companies are banking on RCT results - Cochrane can't do systematic reviews before there's multiple studies, so it's significantly further downstream than what makes pharma money.
What the actual fuck, those are peanuts for any single country or health care system. What a tragedy and a blow to evidence based medicine.
They do mention that Cochrane is only accounting for 7% of systematic reviews these days.
BTW, evidence-based medicine is not the same as science-based medicine. The Placebo Effect is evidence-based medicine.
Often times, you can find cures just by discovering mechanism of action, without PICO/RCT: for example, Helicobacter pylori causing peptic ulcers.
Placebo effect is NOT magic, but we don't understand the mechanism. Good relationship between doctors and patients help recover patients. Patients prefer to see particular doctors, even if the latter prescribe same medications as others, yet the differential effects on patients (when treated by preferred doctors vs others) is non-trivial.
[0] https://en.wikipedia.org/wiki/Category:Drugs_with_unknown_me...
I’m not what 7% of reviews represents, but it seems like particularly dumb austerity measure.
But more importantly Cochrane allows for the pre-registration of such reviews, i.e. where the methodology and goals (hypotheses) are stated before the review is conducted. This is probably the most scientifically rigorous method of assessing evidence, and is not performed for most systematic reviews.
[0] https://www.bma.org.uk/media/6882/bma-ia-juniors-fact-sheet-...
Edit:
For context the thing you linked says UK medical residents (in US language) make $36k/year, and the median household income is also $36k/y[0] .
In the US the average non specialist income is $60k[1], and the median household income is $69k[2]
This tells me things aren’t out of whack.
They also say 40% of residents in the UK want a different job. But residencies are temporary. In a few years they will have a new job - a full doctor.
I’m not saying there’s not a crisis, but that sheet doesn’t explain to me what the crisis actually is. They seem to be doing on par with US residents.
0 https://www.ons.gov.uk/peoplepopulationandcommunity/personal...
1 https://mededits.com/residency-admissions/residency-salary/
UK a consultant earns £100k pretax and pays approximately 30% net tax rate and 60% marginal rate.
What about US?
Apparently family medicine in the US brings in 130% more than the equivalent in the UK (GP which makes up half of all UK doctors).
https://revisingrubies.com/us-vs-uk-doctors-salary/
That's truly astounding.
$230k, which when indexed by median household income, is comparable to the UK at 100k (I assume you used dollars? If not that’s $126k).
Edit: I’m not making any tax statements because taxes are unavoidable and applied to everyone in the nation, so are part of the indexing. That said US taxes sound comparable or more in some areas (taxes in NYC for instance can exceed 50%)
Is that a marginal rate or total tax burden?
According to https://www.moneysavingexpert.com/tax-calculator/ , someone in the UK who earns £100k takes home £67,049:
>Earn £100,000 in 2023/24 and you'll take home £67,049. This means £5,587 in your pocket a month. Over the year you'll pay £27,432 income tax and £5,519 in national insurance.
He didn't last long in London – he was working as an unskilled labourer in the construction industry, the pay was terrible, and the basic necessities of life were so expensive, he could hardly afford to eat. He'd worked similar jobs in Australia and got paid a lot more, and found life more bearable.
Some countries pay high because everything is expensive. Some countries pay low because everything is cheap. The UK seems to be a place where people get paid low because everything is expensive.
What was your brothers take on the US?
He lived in a small town in Oregon with his first wife. While they weren’t paid a lot, life wasn’t expensive either. I think he would have stayed except the marriage didn’t work out. Obviously that isn’t an entirely fair comparison to London, but I still think the pay-to-expenses ratio would be more favourable in small-town US than small-town UK
Summary: https://i.imgur.com/DGT0d0d.png
From: https://www.oecd.org/health/recent-trends-in-international-m...
While it depends on the specialty, in Australia it is not uncommon for consultants to be on AUD 300-400K (= 150-200K GBP, 200-250K USD). I don't understand why salaries in the UK are so low, even for highly educated professionals.
My grandfather was a GP. He hated the NHS so much, he left the UK and never went back. Pay was likely a factor, but he also viewed the NHS as a denial of his professional freedom. While Australia eventually adopted something akin to the NHS (Medicare), I don't think he objected so much to that, since it involved less government control over the how of his job.
Where did your 60% come from?
The net effect is an effective tax rate much higher than the nominal rate.
You lose other benefits to but they aren't factored in as depend on if you have kids or not etc.
What’s the difference between something being scientific and something being evidence-based?
Fundamentally, the mechanism of why a medical intervention works isn't that important if you're thoroughly evaluating outcome measures and controlling confounders.
Science-based: Cholesterol levels are associated with major adverse cardiovascular events (MACE), statins lower cholesterol levels and therefore statin therapy is recommended to reduce MACE.
Evidence-based: A [specifically defined population] was given statin therapy or placebo. We observed lower MACE in the statin group, therefore statin therapy is recommended in [specifically defined population] to reduce MACE.
> statins lower cholesterol levels and therefore
You lost me here - because the science based one is just stating unproven statements to come to some conclusion. If you were to expand on why those two sentences are "proven" you'd likely come to evidence based reasons.
I'm not sure I'm exactly understanding what you're getting at. If you expand on those statements to find direct evidence of MACE (of which there is plenty) then it is an evidence based medicine decision not a science based one.
Please disregard the specific intervention being discussed, it's just an example of the different thought processes and decision making between SBM and EBM. In real life there is plenty of science and evidence behind statins but I'm not intending on addressing reality or why this specific intervention is in use today.
Note the "medicine" part in these refers to the clinical practice of medicine. Maybe this is clearer:
The science based medicine approach (as traditionally defined) only considers the mechanisms and "indirect evidence" when making the clinical recommendation (i.e. statins are proven to lower cholesterol which are strongly correlated with MACE. Cholesterol reduction has also been proven to reduce plaque formation and artery stenosis, therefore we expect MACE will be reduced through statin use and are deciding to administer the intervention).
In this thought process we have not directly measured the outcome of interest (MACE) but we are giving an intervention because it makes sense/through chain of thought, or in other words we are intervening to optimize a variable (serum cholesterol levels) because we have reason to believe this variable reduces MACE based on the mechanism (which may have components proven by evidence like reduction in plaque formation but the primary outcome measure we are discussing, MACE, has not been measured).
In the evidence based medicine approach we are deciding to provide an intervention based on direct measurements of the outcome of interest in a specific population (i.e. less MACE was directly proven) and the decision making process is entirely independent of the mechanism or nature of the intervention (i.e. we are not considering plaque formation and cholesterol levels, simply that statins reduce MACE).
This is effectively the science based method, since it can be tracked to a, perhaps, 'plausible' mechanism (although often now lean towards more 'intriguing' than parsimonious, especially if published somewhere like Nature)
Evidence based medicine has other faults. One subfield is meta-analysis, which looks for very solid observations that are reproducible, but often lacks the complicated modeling that may be necessary to see any highly conditional effects. So the findings tend to be real, but boring and often "well-known". It is however, a good way to check if the "well-known" observations in the literature are reproducible.
Evidence based medicine includes lab/animal studies (i.e. scientific literature in biology) but when we talk about EBM what we really mean is "outcomes based medicine" rather than relying on the science.
EBM reduces the question to "Does intervention X change clinical outcome Y in population Z" to obviate many (but not all) of the clinical translation issues.
"Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial"
https://www.bmj.com/content/363/bmj.k5094
"Conclusions Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention."
Please read the context: https://blogs.bmj.com/bmj/2018/12/13/we-jumped-from-planes-w...
There is even one comment here along that line.
https://www.forbes.com/sites/brucelee/2023/03/11/cochrane-sa...