Medicine is plagued by untrustworthy clinical trials
nature.com
nature.com
This should be an async non blocking evaluation. The statisticians who do it should be anonymous by default. There should be an appeals process for a scientist to explain why an unconventional new method is actually robust.
There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).
Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).
There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.
There should be an independent advisory org that scientists can literally call to ask for advice on structuring the trials. These calls must not be disclosed. Much like farmers can call the government to ask for help on xyz crop problem.
And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.
Any scientist who gets a label in a particular year should be given a vote to review the review agency on several dimensions. These aggregate reviews should be published broadly but not trigger any automatic consequences.
Clear, accessible information is the basis for any self regulating human system. We need more of it in this field.
Rather than outlawing certain ingredients, or creating some kind of health score which a product must be above, Nutrition Facts is a way for suppliers to attest to information about a product in way that is legally binding. If it isn't accurate the penalties are steep.
Consumers then have the information they need to vote with their wallets. Markets cannot function properly without symmetric information, and Nutrition Facts essentially creates a functioning market where one did not exist previously.
Any effort to regulate what's in food would probably be better spent expanding what must be in the Nutrition Facts label. I guess it's nice that we are finally getting around to banning artificial trans fats, but anyone who can read has been able to keep those out of their diet for years. The same can be said about the next bad ingredient, and the one after that.
Mayhap that is overblown? I confess I have not followed it too heavily. Very thankful that I am not allergic to anything in my adult life.
Basically, there were stricter measures put in place called HARPC. From my linked article:
> The new directives mandate that the “Top 8 allergens” identified by FALCPA (peanuts, tree nuts, milk, eggs, wheat, soy, fish, and crustacean shellfish) must either be ingredients of the product and identified as such, or the manufacturer must take extra care (and cost) to ensure that there is no cross-contact with them. There is no middle ground or “out” for the manufacturer, which is why we believe “May contain” type label advisories are heading for extinction. And that poses a problem, at least in the short term.
> companies, when faced with the added burden of instituting and documenting cross-contact prevention measures as dictated by HARPC, may instead choose to add trace amounts of the allergen to the product, as doing so makes the allergen an ingredient of the product and obviates the need for preventative cross-contact measures for that allergen.
> it means that manufacturers will either take stricter measures to prevent cross-contact or add a trace amount of the allergen and list it in the ingredient list, thus eliminating the ambiguity that currently plagues us all.
> A compromise that might have avoided the unintended consequences of companies like Kellogg’s adding traces of allergens to their products is to have offered them a third option: A mandatory “May contain” label advisory for any product made on shared equipment or in shared facilities that did not meet the FSMA threshold for cross-contact prevention. Such label advisories are voluntary today, rendering them ambiguous at best, but a definitively worded and located advisory statement included on all such products would have provided a way for manufacturers to meet the requirements of HARPC without resorting to the addition of allergens.
So I'd be surprised if it was happening on an ongoing basis, but I can definitely see why people would be irked.
I believe the original comment was complaining about the perverse incentive there.
It used to be that companies could slap a "may contain [allergen]" label on things that didn't contain it but were produced in a factory where cross contamination was a possibility. Such labels are *widespread*.
I don't understand the government's incentive in trying to stop this--the actual result was when cross contamination was a possibility the companies reacted by deliberately adding the offending material.
The problem is that it's being looked at in a binary sense. Either it contains the offending material and poses a danger to those affected, or it doesn't and is safe. However, in the real world there's a third population--those who are sensitive to the offending ingredient but not dangerously so. Possibility of cross contamination? That is not going to be a deterrent to me as the worst case outcome is merely unpleasant. Does contain? I'm going to treat it with great skepticism.
All snacks aim to fall at or below a certain number the FDA (or some other agency) put out as being considered a snack. Planet Money did an episode on different M&M varieties having different total weights to account for their different calorie counts. So you get fewer by weight peanut butter M&Ms because they’re more calorie rich
Cochrane (formerly known as the Cochrane Collaboration) is a British international charitable organisation formed to organise medical research findings to facilitate evidence-based choices about health interventions involving health professionals, patients and policy makers.[4][5] It includes 53 review groups that are based at research institutions worldwide. Cochrane has approximately 30,000 volunteer experts from around the world.[6]
The group conducts systematic reviews of health-care interventions and diagnostic tests and publishes them in the Cochrane Library.[7][4]
https://www.cochrane.org/news/statement-physical-interventio...
... which is what I just said: some people got mad at them because their review found no reliable evidence that masking worked (or rather, that mask mandates worked, but these are virtually the same thing).
The null hypothesis for any medical intervention is that it has no effect. You start from that and then try to prove your hypothesis that it does have an effect, which is what medical studies are for. If you can't prove something works then we fall back to the null and assume it doesn't. So that isn't a misleading or inaccurate interpretation of the results, though it would certainly have been politically convenient for the Cochrane organization if their reviewers could have supported the claims of public health authorities.
Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence.
Which I think is definitive.
No it's not the same thing, and that's the key point. If you tell people that masking doesn't work (which is false) then of course mask mandates won't work because adherence will be low. A self-fulfilling prophecy really.
Also health authorities told people masks were highly effective. That's what justified the mandates. So you can't argue mask mandates didn't work because people were told it wouldn't work.
Therefore there's no self fulfilling prophecy here. It didn't even matter what individuals thought anyway, we all had to wear masks.
Although Cochrane much prefers to use RCTs, people have run regressions over the data and there was no link between levels of mask wearing and infection rates. It sucks but it appears that masks just can't stop aerosolized virus, which spreads like a gas. They aren't designed to do that so it's no knock against the manufacturers, who in some cases explicitly warned people that their products would be useless for that purpose (https://pbs.twimg.com/media/EfNmzptXkAEg9Od?format=jpg&name=...).
This is not virtually the same thing. Comparing those two is wildly disingenuous and you know it.
Oh.
I quickly found this:
"The new scientific review on masks and Covid isn’t what you think" Kelsey Piper
https://www.vox.com/future-perfect/2023/2/22/23609499/masks-...
Based on the criticisms, I expect Cochrane will revisit this topic.
Progress isn't a straight line.
For example i might "explain" to you that clean code is about writing the least amount of code possible and you might start code golfing your production systems.
If you want to know what the paper says, read the paper. Journalists are not scientists, most of them do not have the necessary knowledge to understand academic papers, nor do they have an incentive for doing it well. They do have an incentive for generating clicks though, generally by twisting the truth to make things sound more interesting or provoking than they are.
For example, after parsing through the ad hominem attacks and nonsense in that article, their main point is that the Bangladesh study found masks to be effective.
Except that study is junk and all the reported effects were found to be a result of researcher bias [1].
Vox also misrepresents the Danish study, which is probably the best study to date we have on masking effectiveness.
> Progress isn't a straight line.
Yes, but truth is most likely to be found in whatever facts are orthogonal to vox' narrative.
[1] https://trialsjournal.biomedcentral.com/articles/10.1186/s13...
https://en.wikipedia.org/wiki/Ad_hominem
> "Re-analysis on the statistical sampling biases of a mask promotion trial in Bangladesh: a statistical replication"
I'm not remotely qualified to have an opinion.
That said...
The open (public) process as well as the critics sharing their source code is just awesome.
https://trialsjournal.biomedcentral.com/articles/10.1186/s13...
https://github.com/mchikina/maskRCTnote
I share the reviewer's hope that authors of the original study will respond.
https://dailysceptic.org/2023/02/06/dr-carl-heneghan-intervi...
So, a Cochrane review is a study which synthesises all available studies – all that we can find or identity – on a particular topic. It follows a highly structured format and is always preceded by publication of a protocol. All this is to minimise the bias. Also, it is extensively transparent. In this case we are looking at about 300 pages of review. Now, the review called “Physical interventions to interrupt or reduce the spread of respiratory viruses” is called in code A122 for short and I will be using that acronym simply because it is just too long a title. So the protocol was first published in 2006 and then the first version was published in 2007, updated in 2009, 2010, 2011, and then 2020, so this 2023 is the fifth update of this review. And the reason why we update the reviews is they are soon out of date if we don’t do that, especially in some fast moving topics.
This update didn't change the conclusions from any of the prior reviews.
Because masks are so politicized there were numerous attacks on Cochrane this time around, though nobody cared in any of the previous rounds. The Cochrane authors are aware of all the criticisms, but there were no justifications found in any of them to alter the conclusions of the review or their procedures for doing them.
True. But as noted elsethread, Cochrane is not responsible for others misinterpreting the conclusions.
"Statement on 'Physical interventions to interrupt or reduce the spread of respiratory viruses' review" https://www.cochrane.org/news/statement-physical-interventio...
"The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses."
> masks are so politicized
Indeed.
These mistakes will happen from the original scientists, they will happen at the stage of editorial boards, they will happen at peer review, they will happen if external third parties start systematically reviewing every RCT.
So Cochran must similarly be scrutinized for their errors, because they will be making them as well.
And that's even before we get to the political factors outside of science misinterpreting complex data for their own purposes...
It's wicked hard just to get reproducible results (one facet of the replication crisis). Much less the challenges you list.
Confusion and miscommunication is the norm. Rising above that takes Real Effort™.
One of my formative experiences was on a team trying to adopt the processes from the book Applying Use Cases. So simple. Like a recipe. Really, what could be more simple?
We had shared purpose. We all read the book (among others). We discussed. We all thought we were good to go.
And then the wheels fell off once real work started. Turns out we didn't agree. On anything. What is "the system"? What level of abstraction are we working at? What does this line (points at diagram) here mean?
Writing this now, experiencing PTSD flashbacks, I can confidently say I would have never succeeded as a scientist.
No, that was a misinterpretation of the review in the Covid-skeptic sphere. Cochrane have had to issue a statement to clarify : https://www.cochrane.org/news/statement-physical-interventio...
tl;dr : half of the people given masks in these studies didn't wear them consistently or at all, dragging efficacy results down.
Do you want to be the person going around policing mask wearing?
First, some laws probably don't have any positive impact.
Second, there's a difference between accurately summarizing trial results and extrapolating that to the impact of a new law. If there was a death penalty for not wearing masks, perhaps compliance would be better than in the trials and an effect would be shown. This doesn't mean that the trial analysis is wrong, you just can't draw a conclusion about the law from the trial data.
Third, laws have multiple purposes including Justice and Punishment. Some murderers might have zero chance of re-offending but we still want to punish them as a matter of Justice, not because it makes Society safer.
Maybe this is a ridiculous metaphor since public health is not what the justice system is trying to accomplish by imprisoning murderers.
But also, you'll be fine to go on red in the middle of night, where noone is around. Personal responsibility is unfortunately removed by centralised regulations and laws. Thats the problem.
[1] "personal responsibility" makes little sense as a moral catch-all when ignoring your own health can cause other people problems
How much of Netflix cause health problem for you? What about food? How much of junk food are you eating? Or maybe we should ban tabaco and alcohol. Or profesional sport. All of that is potential threat for your health. Being overprotective could damage your mental health... etc.
More regulations = less personal responsibilty.
I will make a comparison, tell me how you find it: "Dumping waste into the river shouldn't be illegal because you dump your waste in your toilet every day and it goes to the river."
Does it sound reasonable? Sure. But I haven't defined scope and I haven't defined effect and I haven't even defined what 'waste' is; I just made up two scenarios which could have similar outcomes and then called it day -- but if you actually care about what happens to the world you need to get a little more detailed than writing silly grandiose political statements and making non-sensical metaphors comparing indicting someone for murder with crafting public health policy.
I generally think laws should be codifications of societal norms. Which also implies that as societal norms change so should laws.
So even things such as murder which people do and we don't want should be codified as illegal. But even if nobody committed murder anymore it should still be illegal as its against societal norms.
I was actually surprised by the mouthwash outcomes as well. Almost no one really talked about mouthwash, but it looked to be useful in the study.
But it's efficacy will really only be decent while it's in your mouth. Once it gets diluted past a certain point, it's not going to be doing anything. You'd probably have similar results with vodka.
No, it wasn't. You should read the paper itself, instead of relying on (sadly) biased editorials about the paper. It literally says what the OP wrote:
> Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence. Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)
The only place they found any plausible signal was comparing N95 respirators against surgical masks, but the evidence was extremely weak:
> We pooled trials comparing N95/P2 respirators with medical/surgical masks (four in healthcare settings and one in a household setting). We are very uncertain on the effects of N95/P2 respirators compared with medical/surgical masks on the outcome of clinical respiratory illness (RR 0.70, 95% CI 0.45 to 1.10; 3 trials, 7779 participants; very low‐certainty evidence). N95/P2 respirators compared with medical/surgical masks may be effective for ILI (RR 0.82, 95% CI 0.66 to 1.03; 5 trials, 8407 participants; low‐certainty evidence). Evidence is limited by imprecision and heterogeneity for these subjective outcomes. The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence).
The editorial you cited was a low point in the history of Cochrane, where they gave in to public outrage and attempted to cast doubt on their own data.
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD...
Whether you think the editorial was them caving or not, they also issued it under their own name with the same weight as their other reviews, so they must have thought enough of it to do so.
Given that there's ample laboratory evidence of the filtering capacity of a good N95 or even a KN95 mask, and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test, I think I'll stick with the mask in future and I have no hesitation recommending winter masking to others who believe they are at risk of complications.
I've liked not being sick for the last three years.
Data is data. Editorials are editorials. The fact that they're published on the same website doesn't change the data. If the Higgs boson was published in the same issue of Physics Letters B as another letter that claimed uncertainty of the result, would you treat them with equal weight?
> and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test
I mean...that's fine? Nobody is telling you what to believe or do. Most of what we do comes without evidence. But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.
> I've liked not being sick for the last three years.
I haven't worn masks and I haven't gotten sick either. Other than Covid -- which I got when we were all wearing masks.
"post hoc, ergo propter hoc."
> But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.
No one's resorting to superstition. You're the one saying there's no value in an intervention that has empiric laboratory evidence to support it. The argument here is what matters at the population level. If the problem is performance, then we train people to select and use masks better, not simply say that there's no point to it at all.
" [Cochrane] found there was no reliable evidence that masks worked against COVID/"
And the editorial: "the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive"
How is "inconclusive" functionally different from "there was no reliable evidence?" Seriously, how do you justify this pedantry while ignoring and obfuscating the truth?
People do much evil by focusing on the wrong facts, the wrong stories, and the wrong lessons learned, while ignoring the right ones. That you are willing to focus on apparently frivolous pedantry while ignoring the fact that so many were forced to use masks without any high-quality scientific evidence that they actually did anything, including children, and all the lessons that should derive from this, is in my opinion, very representative of this type of evil.
The latter sounds like advertising and education about masks rather than wearing the masks themselves. ie telling people to wear masks made no difference in spread probably because people’s minds were already made up about masking.
I din’t see it making any conclusion about masking itself
There are studies that show cheaply made surgical masks shed microplatic fibers which end up in the lungs of those who wear them over prolonged periods.
How will that effect us down the road? TBD.
"We found no reliable evidence that abstinence prevents teen pregnancy"
"We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"
The first is obviously wrong, and if the the second is true it would mean the government should look for other ways to prevent teen pregnancy, but it wouldn't mean that practicing abstinence as an individual doesn't work to prevent pregnancy.
In medicine you cannot distinguish. It is all about the intervention and not about some theoretical best-case scenario.
The intervention is to ask people to wear masks. People comply as they do in real real life and then we measure the results. There was no reliable evidence that this made any noticeable difference.
Now you can change the intervention – instead of asking and mandating masks as we did, we could educate masks wearers more. Unfortunately we have no evidence that it helps.
Perhaps masking could help to an individual wearer? Alas, we didn't collect such evidence either.
Some studies are lab based. In those masks had some effect. But that's not how people use masks in real life, so these results don't mean much.
I think saying "Using X is effective, but only if you actually use X" is obvious. The thing people want to know is "do masks stop the virus" which is an entirely different question from "How many people will wear masks", which is a different question from "What is the effectiveness of interventions to promote mask wearing"
Therefore the real question is how effective is the intervention. It will be (or should be) asked by people responsible with public health policies.
P.S. Cochrane group is not for giving scientific answers to individual people. Its main aim is to evaluate the evidence of different treatments and provide guidance to policy makers and healthcare authorities.
We have similar problems getting schizophrenics to take their meds and getting communities with high rates of open defecation to use toilets, but nobody suggests that we give up on antipsychotics or sanitation facilities.
Every other treatment in medicine including schizophrenia is tested how it works in practice. It is incurable disease and the treatments have many side-effects. Thus the question becomes not “does this medicine cure schizophrenia” but “does this treatment works better than placebo or another treatment?”. When studies are completed, we gather evidence by monitoring real life experience with this treatment.
Medicine is tested according to how it works when people actually take it. People participating in research studies who fail to take their medications (or their placebo for that matter) are kicked from the program and their data is typically discarded entirely.
In fact, often clinical trials are statistically analysed by intention-to-treat, including all people who have been randomised even if they later don't receive the treatment.
Per-protocol-analysis (including only people who follow the study protocol) can also be used but it is more prone to bias.
Besides, with masks it is not simply wearing or not wearing a mask. Even a very diligent mask wearers may wear it in a way that makes it less effective without being aware of that.
In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed. If the real life situation is that most people take medicine in a way that makes it ineffective and so much that the clinical trial cannot find significant effect, then he shouldn't prescribe it. It is just a waste of resources and giving people false hopes.
Fair! That said, intention-to-treat is more likely to greatly underestimate the efficacy of a treatment when non-adherence is expected to be high/isn't being monitored.
> In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed.
I agree, but the solution is to help the patient overcome those barriers not to throw out the medication. It's to give people the information and tools they need to follow the treatment. People who wear masks could be trained on how to properly fit them, take them on/off, store them, replace them, etc. The real life situation around masking included basically none of that. "Wear a mask" was basically all people were told.
It doesn't make sense to fault/dismiss masking if a large part of the population isn't wearing them because they were tricked into believing that masks don't work or that masks will actually make them sick, and another large part of the population wears them, but wasn't shown how to do it correctly.
It's important to be aware that those things are going on within the population, but the next step from there is still "educate the public" and not "abandon all efforts at masking" - at least not until a more accessible alternative which is also as effective as masking becomes available
If we still could not find reliable evidence that masks are effective, then the policy makers should be told that.
There is very little you can do to improve mask wearing technique. We certainly explained these things to doctors, it made no difference in results. If you want to make more controlled studies, you can do that. Don't hold your breath however.
No, we should not continue requiring wearing masks because you are only doing that out of hope. That's not how we do things in medicine. It would be unethical. There are many medicines that show effectiveness in the lab but fail in clinical trials. We don't demand for those medicines to be used until we find more effective alternative. Many unknown factors could cause ineffectiveness in clinical trials, we don't need to understand all of them, just the fact that the drug failed to demonstrate effectiveness and safety in real life settings.
>> a more accessible alternative which is also as effective as masking becomes available
The point is masking was not effective. It has not shown effectiveness anywhere in the real world.
https://jamanetwork.com/journals/jama/fullarticle/2776536
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7191274/
> > "We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"
> The first is obviously wrong,
No. They're equivalent. They both mean "we looked, and we didn't find any confirming evidence." You're confusing "we found no reliable evidence of X" with "we found evidence of NOT X", which is different, and essentially never achievable in empirical studies (note: this is not an invitation to get side-tracked in pedantic debates about proving the null; I'm telling you how actual randomized controlled trials work, in real life.)
Proving a negative via statistics is ~impossible, so what you do instead is to look for significant differences in X, attributable solely or partially to the intervention. If you don't find such a difference (as was the case in the mask review), you say "we found no reliable evidence of X".
But when the Cochrane authors wrote "Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness", they really did mean exactly what it sounds like -- the effect size in an aggregated pool of randomized controlled trials was statistically indistinguishable from zero. You can debate whether or not they looked for the right thing (X), you can debate whether or not adding another big randomized trial would help find X, and so on. But the plain-text interpretation is correct.
Bear in mind a possible source of confusion here: TB bacterium are ~3 microns in size, but viruses are about 0.2 microns. The Cochrane review I mentioned is only about respiratory viruses. So it's possible that they may work against TB but not against flu or COVID.
The NIOSH definition for an N95 is a device able to filter at least 95% of airborne particles that have a mass median aerodynamic diameter of 0.3 micrometers. While SARS-CoV-2 is around 0.1 microns in size, naked COVID-19 viruses in air are rare as they would be torn up nearly immediately, so they are almost always within aerosols. Typical respiratory aerosol range is around half a micron or so [0], and as the aerosol particle size gets smaller, so necessarily must be the amount of virus that is present.
Is this perfect filtration? No, but no one gets sick from a single virus they inhaled either, even with as communicable as the current Omicron variants are. There's a minimum infective dose and they help keep exposure under it.
I've seen it directly--one woman putting on a mask when I approached. The thing is she had been hiking near the back of the pack in a group that got together for the hike. She was at a far higher risk from being downwind of her group (this was not a family bubble) than of me being off to the side.
I can basically guarantee nobody there was experiencing any appreciable symptoms (10,000' up, miles from the cars--not something you're doing with any sort of respiratory infection) but most Covid spread is presymptomatic.
A solo hiker masking when someone approaches makes sense (and is what I did pre-vaccine), but not masking with your group but masking for a stranger? That's merely an illusion of safety and why masks "don't work".
There's also the problem that the Cochrane data included mostly studies of things other than Covid--when you go over their own data only looking at Covid you do see some benefit. Note, also, the pooling of masks and respirators--we already know masks do little against the Omicron variants. Respirators or don't bother.
Cochrane messed up badly in this case by looking at the wrong thing. I'm reminded of the BMJ study showing zero safety benefit from parachutes when jumping from an airplane.
They found only mid-to-low quality evidence supporting the use of masks to prevent ILI. That evidence, for everything but the question of "n95 vs. other", showed an effect size statistically indistinguishable from zero.
You're essentially saying that the error bars on that effect size are big. They are. But they're still centered on zero.
> "There is uncertainty about the effects of face masks. The low to moderate certainty of evidence means our confidence in the effect estimate is limited, and that the true effect may be different from the observed estimate of the effect...There is a need for large, well‐designed RCTs addressing the effectiveness of many of these interventions in multiple settings and populations, as well as the impact of adherence on effectiveness, especially in those most at risk of ARIs."
They admit that they were unclear about it and later were even more explicit.
"Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people's risk of contracting or spreading respiratory viruses."
The review is not able to "address the question" let alone conclude anything about the impact of mask wearing. The review is inconclusive.
OK, fine. If I grant this (I don't, but let's run with it...), it means the following is also true: we mandated something based on such low-quality evidence that no solid conclusions could be made from it.
For example, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8768005/ is "a detailed performance evaluation of the mask is studied from an engineering point of view," which aim to look at how the physics of N95 masks hold up against covid. What the physics shows is that N95 filtration helps block covid particles.
No, there should be prison time for scientists who conduct unethical trials or publish fake results.
The public (and policy makers) place such immense trust in those people and what they publish that nothing less is even remotely adequate.
When someone puts arsenic in food, they go to prison – labeling the food with "contains arsenic" doesn't cut it.
Do this and watch science magically fix itself.
> Ultimately, a lingering question is — as with paper mills — why so many suspect RCTs are being produced in the first place. Mol, from his experiences investigating the Egyptian studies, blames lack of oversight and superficial assessments that promote academics on the basis of their number of publications, as well as the lack of stringent checks from institutions and journals on bad practices.
A substantial part of what's happening here is that first-world countries with generally good cultures of research integrity are basing medical policy on studies done in countries where the system encourages researchers to cheat. British and US authorities can't put Egyptian or Chinese researchers in prison, can they?
I do agree it would be a major deterrent to doing such research, but as it stands you'll get fired for it anyway which is a pretty major deterrent.
Imagine same approach in politics. If some politics put lies in their speeches to manipulate with people, we should call them liers and put them in prison. But somehow it doesn't happen. Looks like society prefer conformity over responsibility.
There is already a mechanism for companies to submit questions to the FDA prior to clinical trial initiation. I'm not in these conversations, but I know the type of questions can be things like: would you accept this endpoint as a proxy for this indication, would you be satisfied with the effect size we expect, and are there other safety concerns you would expect us to evaluate other than those in our current plan. I assume EMA and other regulatory bodies have a similar process, but I'm not positive.
Disclaimer: I work in pharma, but pre-clinically. I am not involved in these clinical or regulatory issues.
Drugs have "prescribing information", referred to in industry as "labeling", which follows a consistent format containing safety, trial results, side effects, and mechanism of action.[1] I recommend people read them for drugs they take.
This should be an async non blocking evaluation. The statisticians who do it should be anonymous by default. There should be an appeals process for a scientist to explain why an unconventional new method is actually robust.
Third party analysis is the main difference here. In the current state, firms run the analysis for FDA review using standard practices, and must explain and get approval for any unconventional methods
>There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).
Labeling contains many relevant numbers. Trial sizes, how many per arm, what was measured, and and final results. Maybe there could be some squishy qualitative summary, but that seems more risky. I would rather know that 1 out of 20 patients died than it got a "2" on the safety scale.
>Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).
Labeling contains drug specific information.
There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.
Drug labeling requires pre-application and and a standard 12 month review period by the FDA prior approval
There should be an independent advisory org that scientists can literally call to ask for advice on structuring the trials. These calls must not be disclosed. Much like farmers can call the government to ask for help on xyz crop problem.
The FDA provides advice on structuring trials and acceptable design, size, power, endpoints. Firms do this by scheduling calls with FDA staticians and experts. [2]
And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.
Maybe there is a difference here. I'm not sure what you mean by punishment? In the current system, The FDA can use the label as "punishment". The FDA may require addition "black box warnings" for drugs that are found to have serious side effects (e.g. high chance of death). They can also pull the label entirely, meaning the drug can not be sold.
Any scientist who gets a label in a particular year should be given a vote to review the review agency on several dimensions. These aggregate reviews should be published broadly but not trigger any automatic consequences.
This is basically how it works for medical device labeling in the EU. There are several "notified bodies" [3] which are private agencies to review the safety and efficacy. The firm then takes their mark of approval to the government agency.
https://www.accessdata.fda.gov/drugsatfda_docs/label/2014/20... https://www.fda.gov/media/72253/download https://climedo.de/en/blog/list-of-mdr-certified-notified-bo...
Generally, reading a paper is much less risky to a person's health than taking a drug so the differences in review process add up meaningfully.
I wonder if we suddenly took 10% of all money spent on science (let's say in medicine) and instead of novel research we used it to redo randomly chosen previous research. Would we loose or gain in terms of new cures? And if we gained, what if we spent 15%, or 25%? That's a great idea for a scientific study to find a point of diminishing returns on "research verification".
Would someone please write a research grant request for this?
Most scientific legwork is done by absolute beginners, i.e., graduate students. They often lack a support structure to focus on what they have learned so far. Most of the world is not Oxbridge, MIT, Stanford.
Where are the beginners' supervisors, you might ask? Chasing the latest trend to secure funding. Pondering how their line of research can be formulated as buzzword-du-jour markov chain. Ass kissing the dean to get department funding.
Having worked in research for 15 years, I am certain about two things: (1) The scientific method yields better results than doing things freehand. (2) Randomly axing 50% of academia would improve the situation.
https://peterattiamd.com/ns001/
https://biolayne.com/reps/how-to-read-research-a-biolayne-gu...
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7392212/
And as a bonus, read books by controversial figures who talk about these challenges through their own published studies and decided it isn't worth fighting.
What would be cool is to see "trust indicators" as part of a study's metadata as it is reviewed through time and continually shared by others. Could be a "study health score" or a checklist showing the study isn't biased heavily by sponsors, methods, or misleading language.
Especially given that medicine is becoming more personal with the advancements of AI and accessibility of tests, someone should be able to understand the health of a study at-a-glance reviewed by peer reviews rather than disclosed only by the authors.
And of course, the criteria for observational-based research differ a lot from mechanism-based, especially since the former can't control for nearly as many variables. The same goes for simulations or interpretive modeling, where experimentation is minimal.
Research studies differ quite a lot in how precisely the mechanism of action is identified, isolated, tested, and results interpreted. IMHO, blackening the trustworthiness of all of science is unhelpful, especially when some models and methods are surely more trustworthy and replicable than others. Better to identify and catalog specific sources of error (or imprecision) in order to remedy them than to just toss the baby.
Whatever. Par for the course in 2023, right?
But this is Nature, a paragon of scientific literature, fueling the distrust of medicine.
There are great reasons to be skeptical of all trials and strengthen peer review and transparency, but this kind of headline is editorial malpractice, in my opinion.
Still, "Carlisle rejected every zombie trial, but by now, almost three years later, most have been published in other journals — sometimes with different data to those submitted with the manuscript he had seen. He is writing to journal editors to alert them, but expects that little will be done." is concerning. I'm almost afraid to know what the list of rejected papers covers.
Still bad, mind you, but unfocused skepticism is its own plague that will cause more trouble.
You can be the sole person in the world with the Bubonic Plague and you'll still be plagued by it. Whether or not untrustworthful clinical trials "cause distress to" [1] Medicine I don't think is debatable; medicine should be based on treatments that have reproducible effects or else people don't get better.
Which is part of my point. If you say that "medicine is plagued" than a natural response is "avoid medicine." But, that is clearly a nonsensical outcome, all told.
We are seemingly in a fifth generation war amongst ourselves for the prizes of attention and public acceptance. Or more succinctly, "being right on the internet".
What may in fact be the next great filter. :)
I would generalize the statement: assuming a reductive order of sciences (medicine->biology->chemistry->physics), if one cannot trust a layer, it seems even more likely that layers above it should be trusted even less.
maybe as an intermediate step we could make available all the recordings to the peer reviewers and only offer the raw experimental data bundled in the paper publicly? maybe in the future we can have 1TB studies without breaking a sweat? maybe all the money we give to publishers can be spent on servers to archive all the primary data so at least we aren't simply filling the pockets of MBAs?
for example, if you have 50 partecipants but only provide the multimedia evidence for 20 of them your study should be thrown out the window
What is the difference (to a layperson) between cherry-picking participants and rejecting participants because they do not meet your study's criteria?
Who funds this agency?
Do the members of the reviewing agency have their own biases, and might those biases tarnish the reputation of a study that is actually well-conducted? (hint: this already happens in journals)
You can see that here: https://kirschsubstack.com/p/pfizer-phase-3-clinical-trial-f...
Pfizer had a trial with 21823 people in the Expirement group and 21823 people in the Placebo group. In the results they excluded data for 1790 from the Expirement group and 1585 Placebo group. However, _crucially_ Pfizer never claimed there were only 100 people in the study after starting with 200; you know pfizer excluded 3375 people because _Pfizer told you_.
Or do subjects need to wave all their doctor-patient privacy rights before joining any trial?
The issue is clearly not the amount of data available to peer reviewers considering it's already easy to detect major flaws in a quarter of published peer reviewed research. The issue is that peer reviewers do a shoddy job which should surprise no one having ever published peer reviewed research.
And to be fair why should they do better? It's generally unpaid, it's poorly paid when it is paid and it's not particularly well considered.
TL;DR: IRBs are a mess, hyper-individualized, and the problem ain't getting any better any time soon.
This is a great approach IMO. Additionally skeptics (for example anti-vaxxers), should be physically present at the trials.
In cognitive science, psychology, even computer science / ai / ml, business.
And the problem with rejected papers getting in somewhere else while being total garbage is pervasive. I've rejected a lot of papers because they were mathematically or statistically bogus only to see them get published elsewhere where reviewers were not so careful (a few times in Nature and Science).
We need an open science movement where you must release everything with your paper. The full pipeline to reconstruct every single result from the raw data. No hiding data. No hiding fmri scans. No "our code only runs on our machine". Etc.
You can just say "anonymize it" but that turns out to be more difficult than it seems initially, especially with many questions of interest.
Also, there's often too many opportunities to do science that is of real public benefit that comes with privacy expectations attached for all kinds of reasons. Cases where there is legitimate consented access but an expectation of privacy without data sharing.
People have tried to solve this problem in different ways (for example, methods where someone can analyze data without having access to it directly) and maybe those solutions will lead to a good resolution. But they often have problems of their own (overhead costs associated with providing anonymized remote data analysis), and don't solve all problems (guarantees of absolutely restricted access to personal data).
Whatever the costs and challenges are, they are not nearly as high as maintaining the status quo.
At the time, no mention was made in the pill’s warning pamphlet.
It is still difficult to secure a class-action suit in America.
Meanwhile, such quality of life would plummet into a silent world, even if one knew American Sign Language fluently beforehand, that tidbit can go against the victim in court.
An interesting point he made was that in the wake of the Thalidomide (https://en.wikipedia.org/wiki/Thalidomide) scandal, the FDA started requiring drugs to be both safe _and effective_
https://www.fda.gov/about-fda/histories-product-regulation/p...
The 'effective' part has proven to be a big source of complexity in the following years, because while it's relatively easy to prove a drug is relatively safe, it's much more difficult and subjective to prove a drug is effective. That closes off a lot of areas of research and development.
The kicker is that Thalidomide was never sold in the USA to begin with.
Anyway, as any reader can imagine, there would be a lot of negative social outcomes to allowing the sale of ineffective drugs. There's a lot of trouble now with medical devices and drugs not being effective, even though we have the rule. I'm not against regulation, I think medical sales are a really complex issue and I don't know how to even judge where the right balance of safety/effectiveness and innovation/freedom could be.
(I point this out because most people only tell the very first part of the thalidomide story).
This should be considered part of every journal's idea of due diligence and this shouldn't be a new idea. Shysters, con artists and snake oil salesmen have been around for a long time. The purpose of a Journal is to publish reliable information and weed out the garbage. How can you do that if you are not looking at the entire picture?
Raw data not available: Ok 97%, Flawed 2%, Zombie 1%
Perhaps it's not good to call an unknown as "Ok"? Maybe Carlisle should add his own paper to the mix?
> This finding alarmed him, too: it suggested that, without access to the IPD — which journal editors usually don’t request and reviewers don’t see — even an experienced sleuth cannot spot hidden flaws.
[0] https://www.science.org/content/blog-post/fakin-it-modern-wa... is just his latest in a long string of well-documented posts.
Of course, this is driven by the money backing the research which, I’m sure, this is why clinical trials in the medical areas can be of a poor quality.
People want to keep their jobs.
I have multiple sclerosis, and I never know if a new medication came out because it's actually better or simply because they could get a new patent for a slightly modified molecule.
Instead of investigating what in the design and execution of the trial led to such a discrepancy, the problem was handled by denying there was a problem, changing the goalposts, reporting ad-hoc hypotheses as facts, silencing all critics, and forcing the public to take the modality anyway or lose jobs, school, and freedom of movement.
I'm confident we will know even more about things as time goes on. I'm less confident on any nefarious motivations in most of it. Reality is that a lot of people died, and everyone was trying to gain control and an advantage over the situation. Mistakes were certainly made, but I am back to low confidence in thinking that everything was a mistake.
Take the Pfizer vaccine. The clinical trial's main endpoint was ~95% efficacy in one thing and one thing only, prevention of symptomatic Covid.
Not reduction in mortality, not reduction in serious disease, not infection, and not spread. The only thing the trial tested and reported was prevention of symptomatic Covid. This is also the sole indication in the package insert as approved by the FDA.
In reality, everyone I know got vaccinated and got symptomatic Covd. I mean everyone, no exceptions. The situation is the similar in my entire country and around the world.
The discrepancy is so massive you don't need large studies. You can easily observe yourself.
(a) Make a survey of the people you know and compare their vaccination status to getting symptomatic Covid. Apply a simple statistical test to test whether it is consistent with the trial results.
[Spolier: it is not]
(b) [advanced] what is your best estimate of the vaccine efficacy given your results of the survey in (a).
The measles vaccine has 95% efficacy. You vaccinate and the disease effectively disappears.
> My understanding was that Omicron came out and basically gave the middle finger to everyone's precautions. With what seemed like literally nothing working against it.
That's an ad-hoc hypothesis.
https://en.wikipedia.org/wiki/Ad_hoc_hypothesis
It was quite clear that the numbers are inconsistent with 95% efficacy way before Omicron.
Folks don't like comparing to the flu, but in this there are obvious similarities. With obviously similar outcomes on the ability of a vaccine to give sterilizing immunity.
Much to your chagrin, though, I actually can say that among my contacts, getting the vaccine basically led to people not getting symptomatic covid. Folks got what they thought of as a bad cold. Almost flu like, but I know very few, if any, folks that were so bad off that they were symptomatic covid. Most wouldn't have even qualified as having a bad flu. (It is frustrating how many folks underestimate how hard the flu hits.)
Contrast with family members that did not get the vaccine in time, and were hospitalized. It was truly different.
Symptomatic Covid is simply a positive Covid test + any flu-like symptoms. What you're describing is symptomatic Covid. This is what was measured and reported in the trial.
You might say that's not very interesting because it doesn't measure anything of importance. You would be right. That is exactly what critics say before the trials.
https://www.bmj.com/content/371/bmj.m4037
The trials were never meant to test whether there would be any mortality benefit, any reduction in serious disease, any reduction in hospitalization, or any effect on infection or transmission.
What they did meausre, turned out to be inconsistent with reality, though.
So, if the concern is you are upset a miracle vaccine didn't get developed, you're losing my interest quick. Anyone that got upset that you had a few symptoms is overblowing concerns to a non-useful degree.
No, the concern is not that a miracle vaccine didn't get developed. The trial measured and reported whether people who got vaccinated got those "few symptoms" vs people who got the placebo. It claimed 95% efficacy in preventing those "few sysmptoms", but it did not do so in reality.
The concern is that the trial results do not agree with reality. That means that something is wrong in either the design or execution of the trial. It's a bug in the trial, and a bug should be debugged.
Seriously, the numbers were drastic for vaccinated versus not in hospitalizations alone. To push the narrative that they were wrong to get vaccines out just feels misguided.
If you are pushing that we should continue to get better at trials and reporting? I agree with that. Any harder push there, though, feels nitpicking at best, and I don't see the direction you are hoping to go.
The vaccine worked pretty well against the Wuhan strain, but Covid breeds variants like it was a rabbit. The farther from the strain coded into the vaccine the less effective the vaccine is. It still seems to be pretty good at reducing the severity, though--the unvaccinated are dying at a far higher rate than the vaccinated.
That's called an ad-hoc hypothesis.
In science and philosophy, an ad hoc hypothesis is a hypothesis added to a theory in order to save it from being falsified.
https://en.wikipedia.org/wiki/Ad_hoc_hypothesis
It could be true, but it is not enough to assert it, it has to be proven.
It was junk science from top to bottom, and this assumes any science was conducted at all. According to a whistle blower, the science was fraudulent.
Am I mistaken in thinking that "bad cold" == symptomatic? Doesn't symptomatic just mean had symptoms? It sounds like you're talking about severe covid.
If my math is correct, thats one 30-something dying for every 1667 infected before vaccines. I don't have hospitalization data handy, but I think "reduced covid" is just what most people had, vaccinated or not. That's not to discount the ones that did get it bad of course, and my condolences for any losses you suffered.
Of course it can still be true that the deaths happened more often in unvaccinated people (did that continue to be true the whole time?), while your individual risk of death was low (the .06 above in my case, and I had a pretty standard cold both times thankfully).
https://www.thelancet.com/journals/lancet/article/PIIS0140-6...
I had what was probably covid early on. Was like the time I got pneumonia. Asthma attacks in my youth were comparable, if much shorter lived. Getting a positive test case later was something that gave me a fever for a few hours. Scary, due to circumstances. But I was back up and moving in basically no time.
Are those rates an argument against the claim that most people didn't have a bad case, vaccinated or not?
> I had what was probably covid early on. Was like the time I got pneumonia. Asthma attacks in my youth were comparable, if much shorter lived. Getting a positive test case later was something that gave me a fever for a few hours. Scary, due to circumstances. But I was back up and moving in basically no time.
How do you know that your possible second case's low severity is due to the vaccine and not the immunity you would have developed in the first case, or weakening of variants (or some mix of all 3), or even just random chance?
It's hard to ignore personal experience, but it only tells us so much. Like me with my 2 unvaccinated cases having an easy time, I'd be remiss if I generalized that to everyone.
You are correct that, if I did, in fact, have an early case of covid, I cannot be sure that the vaccine helped me with the later case. So, as far as that goes, my "evidence" is anecdotal at best and can't be taken fully as proof of anything.
You will have a hard time arguing against vaccines with the aggregate evidence above, though.
I entered the thread at
> getting the vaccine basically led to people not getting symptomatic covid. Folks got what they thought of as a bad cold.
I asked for clarity there because it didn't line up with what I understood to be symptomatic covid (have covid and have any symptoms). It sounded like you were really saying the vaccine led to people in your circle not having severe covid.
I believe it is true that the vaccine reduced instances of severe covid. But my point in this thread is that most people already weren't going to have severe covid (based on ifr rates pre vaccine, though hospitalization data would be more useful here).
In other words, "The rates for vaccinated versus not are a clear indicator that the vaccines helped" is true as I understand it, and not something I'm arguing against. It does not contradict "most cases of covid were not severe, vaccinated or not" though.
Does that make sense?
For specifics in my circle, I really only have my immediate family and some coworkers as direct evidence. Among those, I don't know anyone that got symptomatic anything if they were vaccinated. We had plenty of colds, but only tested positive during a time when that wasn't going through the family. (We only tested due to kid's having contacts that got covid.)
So, to that end, only vaccinated person in the family that ever had symptoms was me. And, as I said, it was super quick. Such that I can't say for sure the kids didn't have symptoms overnight that we just didn't see.
Pulling it back to "most cases overall were not severe," is tough, though. If that is somehow indicative that the vaccines didn't help me, that would also imply that they didn't help the population at large. And the data just doesn't agree with that.
Is that where you are asking? Or did I avoid the question?
And it's easy to see why they would! Given the environment at the time (daily press conferences, scary news articles, demonization of the unvaccinated, mandates) I think it's easy to believe that the vaccine saved you from a death sentence if you get vaccinated and then have an easy case.
It's easy to not notice that in a room of 1667 infected unvaccinated 30 year olds (I don't know how old you are, just using that as an example), maybe over a thousand of them would have had a similar case that you did, and only one of them would have died.
For my part, I suspect it helped me. Childhood asthma and general obesity being what they are. I was almost certainly in elevated risks for my age group. To your point, my age group was still moderate risks, all told.
You cannot simply compare those numbers because the two groups differ in many other respects beyond their vaccination status.
This came out today:
https://www.nejm.org/doi/full/10.1056/NEJMc2306683
Correpondence published in NEJM regarding the landmark study that found that the first booster dose reduced Covid-related mortality by 90%. It turns out that it also reduced non-Covid-related mortality by a similar amount. So either the booster short is a magic elixir that reduces all deaths, or the boosted, as a group, were healthier at the outset.
In the response of the original authors they mention that
> However, boosters were generally not administered to hospitalized patients who were at high risk for death from any cause.
I think you can guess how it affected the hospitalization and death rates of the boosted vs. the unboosted. For some reason, this was not mentioned in the original study.
And the main point mentioned by the original authors in their response:
> However, a strong, unexplained association between the use of the booster and lower mortality not related to Covid-19 remains.
> During the B.1.617.2 (delta) wave in the United States, similar associations were observed between the use of mRNA vaccines and lower mortality not related to Covid-19 and mortality from any cause.
And there's quite a controversy whether the flu vaccine is worthwhile becuase of that. The Cochrane systematic reviews are quite scathing.
> Much to your chagrin, though, I actually can say that among my contacts, getting the vaccine basically led to people not getting symptomatic covid. Folks got what they thought of as a bad cold. Almost flu like, but I know very few, if any, folks that were so bad off that they were symptomatic covid.
That's the definition of symptomatic Covid - a positive Covid test + flu-like symptoms (regardless of severity). That is what the trial measured and reported.
(This is in contrast to Asymptomatic Covid which is a positive Covid test but without any symptoms at all)
> Most wouldn't have even qualified as having a bad flu. (It is frustrating how many folks underestimate how hard the flu hits.)
No one I know experience anything close to a bad flu.
> Contrast with family members that did not get the vaccine in time, and were hospitalized. It was truly different.
Around me it was a mild cold to medium flu regardless of vaccination, including people in their 80s and 90s, with all the pre-existing conditions you can imagine. The only exception was a vaccinated friend (late 40s) who got scary chest pains for several days when he contacted Covid. No treatment beyond Paracetamol and Ibuprofen.
It seems that Pfizer basically rammed the vaccine through because it prevented covid with 95% efficacy for a couple months and made the case that it was too effective to continue the study.
We now know that antibodies from Pfizer decrease significantly and quickly after a couple months, so it seems very likely that Pfizer knew this as well and decided that after two months was the perfect time to conclude their study and start selling vaccines.
To the governments, who have no money but from tax payers.
This I think was the most egregious marketing lie in recent history. That everyone who was jumping up and down for their vaccine was under an impression it was free.
The same people rabbling all day about "transfer of wealth" saw no issue there.
I don't have a stance on covid or vaccines that is terribly unique. But that most people overlooked the massive economic reasons to move in the direction that it did, annoys me.
They ended up tossing one of the intermediate reviews because it was overtaken by events--the objective was met, spend the time on analyzing that data rather than the now-irrelevant intermediate review.
The test did nothing towards establishing how long the protection lasted--they can't have rushed it through based on that being short because they had no measurement of it then.
You simply can't measure time effects in medicine other than by observing them. If you want to know what protection is like after a year you have to wait a year and then measure it. (This is also why we saw repeated changes to the shelf life of the vaccine--the vaccine makers simply didn't have the time to establish what the true shelf life was and thus could only claim what they had measured. Note that this is pervasive in medicine--stored properly most drugs are effective far beyond the stated shelf life. It's just the manufacturers have no reason to spend the money to certify this.)
And in blaming Pfizer you show your bias--why did every vaccine maker do the same thing at the same time??
If anything I'll blame Pfizer for making a weak vaccine. Moderna chose to go with a higher dose that appears to provide slightly more protection at the cost of more side effects at the time.
GET ALL COVID BOOSTERS
WEAR A MASK AT ALL TIMES
VOTE FOR BIDEN FOR 2024, 2028, 2032, 2036
It's not a matter of being nefarious. They (CDC, FDA, health authorities all over the world) really though it was important, but they've used unacceptable means to enforce their beliefs.
Science dies when PR takes over reality.
If reality disagrees with the trial, you have to debug the rial, and find what in the design or exection went wrong.
> Mistakes were certainly made, but I am back to low confidence in thinking that everything was a mistake.
Silencing critics by health authorities is not a mistake. It is an intentional act to enforce your views.
Many pharmaceutical companies have policies which prohibit patients from participating in studies for anti-depressants if they have suicidal ideations. In practice, this significantly reduces the participation of any patient with depression.
I'd urge you to consider following the situations: 1. Prescientific inquiry; and, 2. PreFDA food, drug, and medicine.
Both of those were orders-of-magnitude worse than what we have, now. Could we do better? Sure! Is it broken? No.
And why is it not standard practice to provide anonymous data or even publish the data? What reason exists for that? So that only the researchers them selves can analyze it?
Unfortunately it's not easy to see what the alternatives are, beyond simply not funding research through government/foundation grants. When science is paid for by companies you don't have this incentive issue because the research is judged based on (ultimately) whether it leads to successful products, not whether it leads to lots of papers getting published. You have other incentive issues of course.
> In 2016, the International Committee of Medical Journal Editors (ICMJE), an influential body that sets policy for many major medical titles, had proposed requiring mandatory data-sharing from RCTs. But it got pushback — including over perceived risks to the privacy of trial participants who might not have consented to their data being shared, and the availability of resources for archiving the data. As a result, in the latest update to its guidance, in 2017, it settled for merely encouraging data sharing and requiring statements about whether and where data would be shared.
I'm not a clinician and don't deal with them regularly, but the impression I get is that new studies are published by researchers who have a lot of connections (dubbed thought leaders). They present at conferences. Other clinicians pick up the use case that matches their need (this patient has failed other therapies for this indication, let's try this new thing I'm now aware of). Then as experience grows, clinicians have more nuanced understanding of the use cases for that new information and its reliability. Frustratingly, this can take years, but that's bug that's also a feature.
a) Pharma identities the type of patients they need (e.g., 25-50 female, not pregnant, with specific ailment if Phase III), specific tests, and measurements required throughout the study.
b) pharma contacts third party (3P) to manage study patients.
c) 3P has relationship with dozens or even hundreds of doctors' offices, knowing what office can fulfill the test & measure requirement, and has the potential trial patient pool.
d) 3P has existing contract with these doctors' & hospitals. They get patients onto the study. ( <--- #1 reason this is farmed out in my opinion)
e) Doctors & hospitals perform the study and collect the data.
f) doctors & hospitals pass the data to the 3P
g) 3P passes it to the pharma
h) repeat e) through g) as many times as the study requires it. This can be once, or many times over years.
i) pharma pays 3P, 3P pays doctors & hospitals, and they pay the trial patients - each taking their cut along the way.
There are variations on how this is done, sometimes no 3P, sometimes pharma will have their own pool of patients and 3P. Also this is a very rough flow as there are often checks, audits, and validations (should be) done during the study.
Here's one quick way to rig any clinical trial: Anyone and everyone that has any kind of negative reaction or health condition gets disenrolled. Since it's 'double blind' it appears on the surface level that there's no way to know who's in what group. Naturally, the end result is always the same: The test group had the same number of reactions as the control by the end of the study.
Do you have any sources for this? I'm rather disbelieving of it, but would love to be proven wrong. I can't imagine that _some_ major news outlet wouldn't love to stick it to the status quo with a whistleblower, unless the "whisteblower" made false claims about their proximity in the company to the dangerous/illegal actions they are trying to bring attention to.
We can also see that the FDA does nothing to investigate the integrity of the trials. They just accept whatever the manufacturers tell them.
Each study is different and therefore run differently. Many, many, factors determine how a study is run, analyzed, and published.
Most studies are very small, using only one site and a few volunteers. Most of these never see the light of day, as the results aren't publishable or are uninteresting. Think power law distributions with studies, not normal distributions. These studies are often so small that the collector and interpreter are the same person, typically they are also the grant writer and admin. If lucky, they may get some nurses or undergrads to help out. Again, I'd say this is ~80% of studies.
The really large studies that places like Pfizer run will separate out nearly all parts of a study. So consenters, nurses, intake, data admin, funding admins, stats guys, etc are all different people. These are very expensive studies to run so it's really only for FDA approval, not scientific inquiry and case studies.
Generally, most studies are very small and not publishable. They don't need to separate out everyone. Everyone kinda trusts that everyone else is doing their best. If something snazzy is found, then follow up studies will build on it's findings. Most of the time though, nothing is really found.
These research studies are important (look at how many were conducted on COVID-19 over the last few years) but are typically not held to a particularly high standard, as with most science. Which doesn’t excuse bad data or poor statistics (the latter supposedly supposed to be picked up in peer review).
The amount of record keeping and oversight of a drug approval trial is enormous (and as a consequence insanely expensive) — data handling, having disjoint groups at each stage handling and analyzing data, etc and detailed records of every manufacturing step — think ISO9000 on steroids.
Nobody would bother to go to that effort for a scientific exploration, nor should they. So the bar is much lower.
I am making no excuse for shoddy science! But it is quite unlikely for a licensed drug.
There are multiple pathways to factor in, there is redundancy built in, ie secondary pathways, there is the fact that not all chemicals go where intended (best highlighted by radioactive isotopes). When looking at the history of patented medicine, this really started between WW1 and WW2, before WW1, most GP's prescribed what was found in the body on a like for like basis, and in some cases prescribed organs in various forms, like desiccated thyroid gland for thyroid related problems, for pernicious anaemia, raw liver used to be prescribed to women and so on, but that has its problems like contamination and diseases.
So from WW1/WW2 onwards the rise of patented medicine took hold, but the main problem with patented medicine is the human body hasnt evolved to use these new chemical compounds in the same way as unpatented chemicals which have been around for thousands of years. And todays GP's dont really highlight the side effects of the patented medicine, and because they simply dont ask what you have been eating and drinking etc, they have this hubris which sucker punches your trust and sucks you in like a black hole, until before you know it you are on half a dozen different drugs, your quality of life is going down the pan and you've been left a zombie wondering where did Hitler go wrong modelling the German state of the time on the British Empire? You Americans complaining about the cost of healthcare, should count yourself lucky those insurance companies are looking out for your long term interests and theirs!
Fortunately, hospitals in other countries publishes studies and as english seems to be the main language used for science, I have to tip my hat to the Chinese who are roaring up the charts in terms of investigating and publishing relevant studies that will complement a quality of life one hopes to achieve, and we cant forget Wikipedia, Pubmed and Google for connecting users with pertinent studies.
Saying that I do sometimes wonder if something like ChatGPT has written some studies due to the poor quality of english used, but generally they stand out like a sore thumb.
Anyway, does any know why there is a connection between MDMA, blood clotting and Manganese?
They removed liability from manufacturers, and suddenly the CDC schedule exploded with new products. I mean, chicken pox has a vaccine now?
Just look at how the COVID trials were conducted. They didn't even test each patient. Only some patients that presented symptoms, and then not even all of those patients.
How long did they follow the health outcomes for approved vaccines in the test groups? 3 months at most, and many trials, not even that long. So if someone suffers a neurological condition, well, we just won't know about it.
The was an abundance of evidence that the covid vaccines had a reasonable likelihood of being unsafe. Every single previous attempt at a coronavirus vaccine had failed, sometimes catastrophically (killing all the test animals), that's why there wasn't an existing coronavirus vaccine on the market. Every single previous attempt to bring a mRNA treatment to the mass-market had failed due to safety issues. Even in the Pfizer vaccine trial there were overall more deaths in the vaccinated group than the placebo group, due to cardiac deaths (although it wasn't a statistically significant enough amount to draw a conclusion, it does demonstrate that the trial had no power to identify if the vaccine was net-harmful, as it didn't have enough participants to make a meaningful conclusion about the effect of the vaccine on excess deaths).
Not a single prototype mRNA-based drug passed phase3 trials at any point - right up until the multiple ones within a month of each other were deployed globally.
The massive and remarkable coincidence of that, is truly a special moment in history.
The pregnancy trials were outright abandoned.
They didn't even conduct clinical trials for the bivalent boosters.
Zero efficacy in children, yet still strongly recommended by the media and the state.
I have a lot of work to do ahead of me, researching all of these vaccines for my kids. Makes my head hurt just thinking about it.
You can’t trust pharma companies if their data is secret.
here you go anonuser123456
You will likely be able to find scientific studies toward any conclusion you want to make but there is objective evidence of this and in my own personal life significant anecdotal evidence. Given the OP title and topic and the massive reproducibility problem in medicine and perverse incentive problem in medicine, you will not be able to know the truth about the safety of medications to a high degree.
It is like the reviews problem on Amazon. All medications come with studies giving 5 star reviews. But the reality is nowhere near the same. Blackbox warnings and such are all we have to place a significant ding on this problem before taking the drugs off the market for clearly causing intolerable issues in large swaths of people.
This is a complex muddled issue. Argumentation with citations of 'articles' from journals likely will not lead to truthful discovery, unfortunately.
https://peterattiamd.com/why-a-recent-study-hasnt-shaken-my-...
I've not heard that claim but am interested. Overprescripton certainly doesn't require that the trials were bad in any way.
And I'm also curious about this idea of overprescription, because I hear it sometimes from extremely political people but have never heard it from scientists (and scientists are always trying to find some way to critique current practice, so that statins don't rise to that level is a surprise to me).
Media and gov, or media at the behest of gov (as the twitter files prove, but it was obvious without them), censored as much as thet could, everything that went against what they wanted to push. It wasn't science but security/hygiene theater and it worked, because people did go along and did turn on their neighbours who opposed the measures.
But now we slowly get tidbits of things we can debate again ... Funny that, but no recognition that the entire lockdown, masks and vaccine mandate effort + economic destruction and theft (tax money to corps as "aid") was never a reasonable, logical or scientific response. It was an authoritarian and corrupt response of extreme Effectiveness and cynicism.
But here comes some guy to say "people have always quarantined in pandemics" or some other disingenuous claim that ignores the reality of what happened: healthy people denied basic human and constitutional rights.
And note the lessons of history:
1) There will always be those who choose short term economic interests over safety when the threat isn't absolutely proven. They'll always close the barn door too late.
2) Places that take epidemic/pandemic threats seriously tend to fare better economically in the long run.
The risk profile for people was always ridículous and you were never going to contain it once it was widespread, which it was, but somehow we believed in the rolling "it's just 2 weeks".
1) it wasn't short term. It wasn't safety. Short and long term you hurt the vulnerable and you helped the rich and powerful.
2) Rich places will keep being richer and poor places poorer, and when the powers that be decided that enough was enough, all the concerns of the hypochondriacs suddenly didn't matter.
You could tell, if you paid attention, that politicians weren't afraid after the initial surge, but wanted "the masses" to be. You even have definite proof in many places, one of them being UK and number 10. Can't link it now, sorry. But Google downing Street covid rules or something on that note and you can probably find a lot. It wasn't limited to the UK. It was everywhere you looked properly.
You were scammed and you either were well off and didn't mind that much or you want to pretend you weren't for your own mental health. Because the truth is a hard pill to swallow.
And it's not a matter of wanting the masses to be scared. Rather, the pattern I have seen again and again of people treating those they regard as good people as safe. That's been the huge chink in the armor against Covid all along. Doctors mostly got infected from coworkers, not patients because they were careful around patients and let down their guard in non-patient areas.
We always knew how Natural immunizaty worked and Omicron was easy to predict, but in 2020 it was a conspiracy theory to mention natural immunity.
> The end result was approximately halving the death toll, plus delaying a lot of infections until after the vaccine was available also cut the death toll considerably. All in all millions of lives were saved by the measures you shun
The halved death toll is a beautiful fabrication of yours. not at all supported by looking at excess deaths, which is another analysis that was frowned upon and ignored while implementing Draconian measures.
You didn't really saved much, you pretend you did and the excesss deaths are now more, funny that. Also, all the suicides, backlogged patients who didn't get treatment in time (cancer deaths) and the indirect deaths and lowered quality of life due to economic impact that disproportionately affects the least wealthy, that's all ignored for the hypothetical "my elephant repellent worked".
> And it's not a matter of wanting the masses to be scared.
WhatsApp messages between politicians and very obvious propaganda (see NHS) Zwith people on respirators say otherwise.
>Rather, the pattern I have seen again and again of people treating those they regard as good people as safe. That's been the huge chink in the armor against Covid all along. Doctors mostly got infected from coworkers, not patients because they were careful around patients and let down their guard in non-patient areas.
I don't even know what you're talking about here.
"Letting the guard down" was always a ridiculous thing when your "guard up" is unscientific and authoritarian hygiene theater. Your politicians didn't care for it but would send the police so you would.