Hacker News Reader

Comment by obblekk | Hacker News Reader

There should be Nutrition Facts but for scientific trials. Independent agency just publishing quality assessments of the trial.

This should be an async non blocking evaluation. The statisticians who do it should be anonymous by default. There should be an appeals process for a scientist to explain why an unconventional new method is actually robust.

There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).

Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).

There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.

There should be an independent advisory org that scientists can literally call to ask for advice on structuring the trials. These calls must not be disclosed. Much like farmers can call the government to ask for help on xyz crop problem.

And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.

Any scientist who gets a label in a particular year should be given a vote to review the review agency on several dimensions. These aggregate reviews should be published broadly but not trigger any automatic consequences.

Clear, accessible information is the basis for any self regulating human system. We need more of it in this field.

View on HN

There's the Cochrane Collaboration. They don't tick off every item on your list but it's fairly close to what you're asking for. It's mentioned in the article as they do a lot of meta-studies. Unfortunately they only started trying to spot fraudulent RCTs in 2021. Also in recent times some people don't like them, because they did a big review of mask studies and found there was no reliable evidence that masks worked against COVID.

Cochrane (formerly known as the Cochrane Collaboration) is a British international charitable organisation formed to organise medical research findings to facilitate evidence-based choices about health interventions involving health professionals, patients and policy makers.[4][5] It includes 53 review groups that are based at research institutions worldwide. Cochrane has approximately 30,000 volunteer experts from around the world.[6]

The group conducts systematic reviews of health-care interventions and diagnostic tests and publishes them in the Cochrane Library.[7][4]

https://en.wikipedia.org/wiki/Cochrane_(organisation)

"Many commentators have claimed that a recently-updated Cochrane Review shows that 'masks don't work', which is an inaccurate and misleading interpretation."

https://www.cochrane.org/news/statement-physical-interventio...

Next sentence: "It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive."

... which is what I just said: some people got mad at them because their review found no reliable evidence that masking worked (or rather, that mask mandates worked, but these are virtually the same thing).

The null hypothesis for any medical intervention is that it has no effect. You start from that and then try to prove your hypothesis that it does have an effect, which is what medical studies are for. If you can't prove something works then we fall back to the null and assume it doesn't. So that isn't a misleading or inaccurate interpretation of the results, though it would certainly have been politically convenient for the Cochrane organization if their reviewers could have supported the claims of public health authorities.

That sentence doesn't say what you think it says. It says "interventions to promote mask wearing". That's not mask wearing, it's telling people to wear masks. It is both true that wearing masks helps and that it's hard to tell if promoting mask-wearing changed enough behavior to matter. Mostly, those interventions do nothing.

That's an ambiguous sentence. The main results of the study conclude:

Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence.

Which I think is definitive.

The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize.

> no reliable evidence that masking worked(or rather, that mask mandates worked, but these are virtually the same thing).

No it's not the same thing, and that's the key point. If you tell people that masking doesn't work (which is false) then of course mask mandates won't work because adherence will be low. A self-fulfilling prophecy really.

Compliance for COVID mask mandates was measured and found to be extremely high, especially at the start (>95%). These mandates were enforced by harsh penalties so high compliance levels is no surprise. Thus you can't argue mask mandates didn't work because of low compliance.

Also health authorities told people masks were highly effective. That's what justified the mandates. So you can't argue mask mandates didn't work because people were told it wouldn't work.

Therefore there's no self fulfilling prophecy here. It didn't even matter what individuals thought anyway, we all had to wear masks.

Although Cochrane much prefers to use RCTs, people have run regressions over the data and there was no link between levels of mask wearing and infection rates. It sucks but it appears that masks just can't stop aerosolized virus, which spreads like a gas. They aren't designed to do that so it's no knock against the manufacturers, who in some cases explicitly warned people that their products would be useless for that purpose (https://pbs.twimg.com/media/EfNmzptXkAEg9Od?format=jpg&name=...).

> ... which is what I just said: some people got mad at them because their review found no reliable evidence that masking worked (or rather, that mask mandates worked, but these are virtually the same thing).

This is not virtually the same thing. Comparing those two is wildly disingenuous and you know it.

Yes because a null hypothesis cannot be proven. Basic science.

Nothing can be proven in science- only in math.

> Also in recent times some people don't like them, because they did a big review of mask studies and found there was no reliable evidence that masks worked against COVID.

Oh.

I quickly found this:

"The new scientific review on masks and Covid isn’t what you think" Kelsey Piper

https://www.vox.com/future-perfect/2023/2/22/23609499/masks-...

Based on the criticisms, I expect Cochrane will revisit this topic.

Progress isn't a straight line.

Vox is an awful source.

In this case it's the source of nothing more than an explanation of the study which we are already discussing.

"An explanation" can be wildly misleading.

For example i might "explain" to you that clean code is about writing the least amount of code possible and you might start code golfing your production systems.

If you want to know what the paper says, read the paper. Journalists are not scientists, most of them do not have the necessary knowledge to understand academic papers, nor do they have an incentive for doing it well. They do have an incentive for generating clicks though, generally by twisting the truth to make things sound more interesting or provoking than they are.

Exactly. My guess is the person making statements about the study that aren't supported by it to begin with also just read some bad explanations of it. This is not worse. Criticising widespread faulty "interpretations" and pointing out it doesn't say what all articles say seems way more likely to get someone to look at the source (especially when engaged in online arguments) instead of using articles to back their claims up.

Vox? I would recommend seeking elsewhere for truth.

For example, after parsing through the ad hominem attacks and nonsense in that article, their main point is that the Bangladesh study found masks to be effective.

Except that study is junk and all the reported effects were found to be a result of researcher bias [1].

Vox also misrepresents the Danish study, which is probably the best study to date we have on masking effectiveness.

> Progress isn't a straight line.

Yes, but truth is most likely to be found in whatever facts are orthogonal to vox' narrative.

[1] https://trialsjournal.biomedcentral.com/articles/10.1186/s13...

> parsing through the ad hominem attacks

https://en.wikipedia.org/wiki/Ad_hominem

> "Re-analysis on the statistical sampling biases of a mask promotion trial in Bangladesh: a statistical replication"

I'm not remotely qualified to have an opinion.

That said...

The open (public) process as well as the critics sharing their source code is just awesome.

https://trialsjournal.biomedcentral.com/articles/10.1186/s13...

https://github.com/mchikina/maskRCTnote

I share the reviewer's hope that authors of the original study will respond.

Cochrane revisit topics from time to time to update their reviews as new studies appear. The question of mask effectiveness was reviewed in the past also. There's an interview with one of the authors of this round's review here:

https://dailysceptic.org/2023/02/06/dr-carl-heneghan-intervi...

So, a Cochrane review is a study which synthesises all available studies – all that we can find or identity – on a particular topic. It follows a highly structured format and is always preceded by publication of a protocol. All this is to minimise the bias. Also, it is extensively transparent. In this case we are looking at about 300 pages of review. Now, the review called “Physical interventions to interrupt or reduce the spread of respiratory viruses” is called in code A122 for short and I will be using that acronym simply because it is just too long a title. So the protocol was first published in 2006 and then the first version was published in 2007, updated in 2009, 2010, 2011, and then 2020, so this 2023 is the fifth update of this review. And the reason why we update the reviews is they are soon out of date if we don’t do that, especially in some fast moving topics.

This update didn't change the conclusions from any of the prior reviews.

Because masks are so politicized there were numerous attacks on Cochrane this time around, though nobody cared in any of the previous rounds. The Cochrane authors are aware of all the criticisms, but there were no justifications found in any of them to alter the conclusions of the review or their procedures for doing them.

> This update didn't change the conclusions from any of the prior reviews. ... The Cochrane authors are aware of all the criticisms, but there were no justifications found in any of them to alter the conclusions of the review or their procedures for doing them.

True. But as noted elsethread, Cochrane is not responsible for others misinterpreting the conclusions.

"Statement on 'Physical interventions to interrupt or reduce the spread of respiratory viruses' review" https://www.cochrane.org/news/statement-physical-interventio...

"The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses."

> masks are so politicized

Indeed.

If science is going to be "self-correcting" then it has to make mistakes in the first place.

These mistakes will happen from the original scientists, they will happen at the stage of editorial boards, they will happen at peer review, they will happen if external third parties start systematically reviewing every RCT.

So Cochran must similarly be scrutinized for their errors, because they will be making them as well.

And that's even before we get to the political factors outside of science misinterpreting complex data for their own purposes...

Yes and:

It's wicked hard just to get reproducible results (one facet of the replication crisis). Much less the challenges you list.

Confusion and miscommunication is the norm. Rising above that takes Real Effort™.

One of my formative experiences was on a team trying to adopt the processes from the book Applying Use Cases. So simple. Like a recipe. Really, what could be more simple?

We had shared purpose. We all read the book (among others). We discussed. We all thought we were good to go.

And then the wheels fell off once real work started. Turns out we didn't agree. On anything. What is "the system"? What level of abstraction are we working at? What does this line (points at diagram) here mean?

Writing this now, experiencing PTSD flashbacks, I can confidently say I would have never succeeded as a scientist.

> they did a big review of mask studies and found there was no reliable evidence that masks worked against COVID.

No, that was a misinterpretation of the review in the Covid-skeptic sphere. Cochrane have had to issue a statement to clarify : https://www.cochrane.org/news/statement-physical-interventio...

tl;dr : half of the people given masks in these studies didn't wear them consistently or at all, dragging efficacy results down.

Which is actually good proof that _requiring_ people to wear masks doesn’t help. Mask mandates are pointless even when masks are useful.

By that logic why make any laws? Why make murder illegal if some people are going to kill anyways?

That is why we have police officers to try to stop the people who have proven they are willing to murder.

Do you want to be the person going around policing mask wearing?

Three points.

First, some laws probably don't have any positive impact.

Second, there's a difference between accurately summarizing trial results and extrapolating that to the impact of a new law. If there was a death penalty for not wearing masks, perhaps compliance would be better than in the trials and an effect would be shown. This doesn't mean that the trial analysis is wrong, you just can't draw a conclusion about the law from the trial data.

Third, laws have multiple purposes including Justice and Punishment. Some murderers might have zero chance of re-offending but we still want to punish them as a matter of Justice, not because it makes Society safer.

Presumably less people kill in that case.

Would you kill of that would be legal?

If I did decide to, and it was your spouse/child/parent/sibling, would you then kill me in revenge?

Maybe this is a ridiculous metaphor since public health is not what the justice system is trying to accomplish by imprisoning murderers.

Point was that some questions are not about law. Im sure you are not going on red just because you could get fined for it.

But also, you'll be fine to go on red in the middle of night, where noone is around. Personal responsibility is unfortunately removed by centralised regulations and laws. Thats the problem.

In reality things are complicated and one principle[1] cannot account for all situations, so we deal with things differently as they come up. Murder happens to be an extremely old problem; public health is also, but knowledge of microbes and infection mechanisms are brand new, so we are still figuring it out.

[1] "personal responsibility" makes little sense as a moral catch-all when ignoring your own health can cause other people problems

- ...ignoring your own health can cause other people problems"

How much of Netflix cause health problem for you? What about food? How much of junk food are you eating? Or maybe we should ban tabaco and alcohol. Or profesional sport. All of that is potential threat for your health. Being overprotective could damage your mental health... etc.

More regulations = less personal responsibilty.

Your comparisons between issues in modern cultural and political concepts are fascinating. Please tell me how overeating, obesity, personal addictions, competitive athletics and media consumption are related to microbial infection vectors so that I can write a definitive thesis about externalities and how they don't exist because I want to live in a libertarian fantasy world.

I will make a comparison, tell me how you find it: "Dumping waste into the river shouldn't be illegal because you dump your waste in your toilet every day and it goes to the river."

Does it sound reasonable? Sure. But I haven't defined scope and I haven't defined effect and I haven't even defined what 'waste' is; I just made up two scenarios which could have similar outcomes and then called it day -- but if you actually care about what happens to the world you need to get a little more detailed than writing silly grandiose political statements and making non-sensical metaphors comparing indicting someone for murder with crafting public health policy.

I'm just reply to your statements with questions and you refuse to answer by creating new conclusions.

I don't see any questions except the murder one (which I noted is non-sensical) and some rhetorical ones.

Well depends on who you ask.

I generally think laws should be codifications of societal norms. Which also implies that as societal norms change so should laws.

So even things such as murder which people do and we don't want should be codified as illegal. But even if nobody committed murder anymore it should still be illegal as its against societal norms.

This is important to point out.

I was actually surprised by the mouthwash outcomes as well. Almost no one really talked about mouthwash, but it looked to be useful in the study.

Mouthwash is typically alcohol based. Alcohol is a pretty good disinfectant in general.

But it's efficacy will really only be decent while it's in your mouth. Once it gets diluted past a certain point, it's not going to be doing anything. You'd probably have similar results with vodka.

I know doctors who recommended drinking whiskey early in the pandemic for that reason (and also the usual reasons people dealing with trauma reach for whiskey)

That link agrees with what GP actually said: "no reliable evidence".

I'm mass transit where masks will required, most of the people wore the masks covering their chin. I believe the research is accurate and that saying that masking up more people doesn't work because they won't wear them properly

> that was a misinterpretation of the review in the Covid-skeptic sphere.

No, it wasn't. You should read the paper itself, instead of relying on (sadly) biased editorials about the paper. It literally says what the OP wrote:

> Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence. Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)

The only place they found any plausible signal was comparing N95 respirators against surgical masks, but the evidence was extremely weak:

> We pooled trials comparing N95/P2 respirators with medical/surgical masks (four in healthcare settings and one in a household setting). We are very uncertain on the effects of N95/P2 respirators compared with medical/surgical masks on the outcome of clinical respiratory illness (RR 0.70, 95% CI 0.45 to 1.10; 3 trials, 7779 participants; very low‐certainty evidence). N95/P2 respirators compared with medical/surgical masks may be effective for ILI (RR 0.82, 95% CI 0.66 to 1.03; 5 trials, 8407 participants; low‐certainty evidence). Evidence is limited by imprecision and heterogeneity for these subjective outcomes. The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence).

The editorial you cited was a low point in the history of Cochrane, where they gave in to public outrage and attempted to cast doubt on their own data.

https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD...

From the editorial: "It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive. Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people's risk of contracting or spreading respiratory viruses."

Whether you think the editorial was them caving or not, they also issued it under their own name with the same weight as their other reviews, so they must have thought enough of it to do so.

Given that there's ample laboratory evidence of the filtering capacity of a good N95 or even a KN95 mask, and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test, I think I'll stick with the mask in future and I have no hesitation recommending winter masking to others who believe they are at risk of complications.

I've liked not being sick for the last three years.

Yup, we lead lives where it's simply not that big an issue to protect ourselves. While I think my chance of dying from getting it would be very low the issue of long term damage is another matter--it certainly looks to me like it damages everybody, just not always to the point they notice. The damage is probably cumulative.

> they also issued it under their own name with the same weight as their other reviews, so they must have thought enough of it to do so.

Data is data. Editorials are editorials. The fact that they're published on the same website doesn't change the data. If the Higgs boson was published in the same issue of Physics Letters B as another letter that claimed uncertainty of the result, would you treat them with equal weight?

> and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test

I mean...that's fine? Nobody is telling you what to believe or do. Most of what we do comes without evidence. But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.

> I've liked not being sick for the last three years.

I haven't worn masks and I haven't gotten sick either. Other than Covid -- which I got when we were all wearing masks.

"post hoc, ergo propter hoc."

That's drawing an unnecessarily sharp description. To a first approximation all Cochrane pieces are editorials. They're interpreting what's actually out there.

> But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.

No one's resorting to superstition. You're the one saying there's no value in an intervention that has empiric laboratory evidence to support it. The argument here is what matters at the population level. If the problem is performance, then we train people to select and use masks better, not simply say that there's no point to it at all.

Seriously, what's the difference between what OP wrote:

" [Cochrane] found there was no reliable evidence that masks worked against COVID/"

And the editorial: "the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive"

How is "inconclusive" functionally different from "there was no reliable evidence?" Seriously, how do you justify this pedantry while ignoring and obfuscating the truth?

People do much evil by focusing on the wrong facts, the wrong stories, and the wrong lessons learned, while ignoring the right ones. That you are willing to focus on apparently frivolous pedantry while ignoring the fact that so many were forced to use masks without any high-quality scientific evidence that they actually did anything, including children, and all the lessons that should derive from this, is in my opinion, very representative of this type of evil.

"Many were forced" != "there's no value"

I’m reading these as completely different.

The latter sounds like advertising and education about masks rather than wearing the masks themselves. ie telling people to wear masks made no difference in spread probably because people’s minds were already made up about masking.

I din’t see it making any conclusion about masking itself

We may not be sure that masks help, but we're completely sure that they don't hurt so I don't see the problem personally.

I'm completely sure masks hurt my pocketbook and my ability to keep my car tidy, and that forcing people to mask has additional costs. There are cost/benefit questions that aren't as trivial as you imply, and they should be made based on reliable data.

Absolutely false. There are lots of negatives to mask wearing, starting with inducing developmental problems in children and continuing on with massive increases in long lasting trash and then into more speculative issues with breathing. It's not a harmless activity.

I wouldn’t be so sure they don’t hurt.

There are studies that show cheaply made surgical masks shed microplatic fibers which end up in the lungs of those who wear them over prolonged periods.

How will that effect us down the road? TBD.

It's not "inconclusive" and "there was no reliable evidence" that are different, it's the promoting part that makes them completely different.

"We found no reliable evidence that abstinence prevents teen pregnancy"

"We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"

The first is obviously wrong, and if the the second is true it would mean the government should look for other ways to prevent teen pregnancy, but it wouldn't mean that practicing abstinence as an individual doesn't work to prevent pregnancy.

Cochrane review doesn't make this distinction.

In medicine you cannot distinguish. It is all about the intervention and not about some theoretical best-case scenario.

The intervention is to ask people to wear masks. People comply as they do in real real life and then we measure the results. There was no reliable evidence that this made any noticeable difference.

Now you can change the intervention – instead of asking and mandating masks as we did, we could educate masks wearers more. Unfortunately we have no evidence that it helps.

Perhaps masking could help to an individual wearer? Alas, we didn't collect such evidence either.

Some studies are lab based. In those masks had some effect. But that's not how people use masks in real life, so these results don't mean much.

> But that's not how people use masks in real life, so they don't mean much.

I think saying "Using X is effective, but only if you actually use X" is obvious. The thing people want to know is "do masks stop the virus" which is an entirely different question from "How many people will wear masks", which is a different question from "What is the effectiveness of interventions to promote mask wearing"

The first question is pointless for someone responsible for public health. People want the answer to it because they don't want to think about all these related issues and have simplistic idea that they can protect themselves. But chances are their compliance is exactly the same as among people in those studies.

Therefore the real question is how effective is the intervention. It will be (or should be) asked by people responsible with public health policies.

P.S. Cochrane group is not for giving scientific answers to individual people. Its main aim is to evaluate the evidence of different treatments and provide guidance to policy makers and healthcare authorities.

If you are responsible for public health and the answer to the first question is "no" then you have no need to ask the other two. Figuring out what we can do to get people to do what works is important too, but it's not the only thing that matters. People can be educated and their habits changed.

We have similar problems getting schizophrenics to take their meds and getting communities with high rates of open defecation to use toilets, but nobody suggests that we give up on antipsychotics or sanitation facilities.

The first answer is too vague to have a meaningful answer in case.

Every other treatment in medicine including schizophrenia is tested how it works in practice. It is incurable disease and the treatments have many side-effects. Thus the question becomes not “does this medicine cure schizophrenia” but “does this treatment works better than placebo or another treatment?”. When studies are completed, we gather evidence by monitoring real life experience with this treatment.

> Every other treatment in medicine including schizophrenia is tested how it works in practice.

Medicine is tested according to how it works when people actually take it. People participating in research studies who fail to take their medications (or their placebo for that matter) are kicked from the program and their data is typically discarded entirely.

That is generally not true.

In fact, often clinical trials are statistically analysed by intention-to-treat, including all people who have been randomised even if they later don't receive the treatment.

Per-protocol-analysis (including only people who follow the study protocol) can also be used but it is more prone to bias.

Besides, with masks it is not simply wearing or not wearing a mask. Even a very diligent mask wearers may wear it in a way that makes it less effective without being aware of that.

In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed. If the real life situation is that most people take medicine in a way that makes it ineffective and so much that the clinical trial cannot find significant effect, then he shouldn't prescribe it. It is just a waste of resources and giving people false hopes.

> In fact, often clinical trials are statistically analysed by intention-to-treat

Fair! That said, intention-to-treat is more likely to greatly underestimate the efficacy of a treatment when non-adherence is expected to be high/isn't being monitored.

> In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed.

I agree, but the solution is to help the patient overcome those barriers not to throw out the medication. It's to give people the information and tools they need to follow the treatment. People who wear masks could be trained on how to properly fit them, take them on/off, store them, replace them, etc. The real life situation around masking included basically none of that. "Wear a mask" was basically all people were told.

It doesn't make sense to fault/dismiss masking if a large part of the population isn't wearing them because they were tricked into believing that masks don't work or that masks will actually make them sick, and another large part of the population wears them, but wasn't shown how to do it correctly.

It's important to be aware that those things are going on within the population, but the next step from there is still "educate the public" and not "abandon all efforts at masking" - at least not until a more accessible alternative which is also as effective as masking becomes available

The population was told that masks certainly work, in certain areas mandates made sure that compliance is very high >95%.

If we still could not find reliable evidence that masks are effective, then the policy makers should be told that.

There is very little you can do to improve mask wearing technique. We certainly explained these things to doctors, it made no difference in results. If you want to make more controlled studies, you can do that. Don't hold your breath however.

No, we should not continue requiring wearing masks because you are only doing that out of hope. That's not how we do things in medicine. It would be unethical. There are many medicines that show effectiveness in the lab but fail in clinical trials. We don't demand for those medicines to be used until we find more effective alternative. Many unknown factors could cause ineffectiveness in clinical trials, we don't need to understand all of them, just the fact that the drug failed to demonstrate effectiveness and safety in real life settings.

>> a more accessible alternative which is also as effective as masking becomes available

The point is masking was not effective. It has not shown effectiveness anywhere in the real world.

There is very much plenty of fairly reliable evidence that masks work. And the better the compliance the better they work. In nurse studies you get much better results than in population studies, for instance. Now that I'm looking I'm hard pressed to find any studies that go against this conclusion.

https://jamanetwork.com/journals/jama/fullarticle/2776536

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7191274/

https://onlinelibrary.wiley.com/doi/full/10.1111/jocn.15401

https://bmjopen.bmj.com/content/5/4/e006577

Have you looked at Cochrane review?

> > "We found no reliable evidence that abstinence prevents teen pregnancy"

> > "We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"

> The first is obviously wrong,

No. They're equivalent. They both mean "we looked, and we didn't find any confirming evidence." You're confusing "we found no reliable evidence of X" with "we found evidence of NOT X", which is different, and essentially never achievable in empirical studies (note: this is not an invitation to get side-tracked in pedantic debates about proving the null; I'm telling you how actual randomized controlled trials work, in real life.)

Proving a negative via statistics is ~impossible, so what you do instead is to look for significant differences in X, attributable solely or partially to the intervention. If you don't find such a difference (as was the case in the mask review), you say "we found no reliable evidence of X".

But when the Cochrane authors wrote "Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness", they really did mean exactly what it sounds like -- the effect size in an aggregated pool of randomized controlled trials was statistically indistinguishable from zero. You can debate whether or not they looked for the right thing (X), you can debate whether or not adding another big randomized trial would help find X, and so on. But the plain-text interpretation is correct.

The review found very few studies into the effectiveness of N95/respirators against ILIs, and from those studies they concluded "wearing N95/P2 respirators probably makes little to no difference".

Bear in mind a possible source of confusion here: TB bacterium are ~3 microns in size, but viruses are about 0.2 microns. The Cochrane review I mentioned is only about respiratory viruses. So it's possible that they may work against TB but not against flu or COVID.

I'm pretty aware of how large a TB bacillus is, thanks.

The NIOSH definition for an N95 is a device able to filter at least 95% of airborne particles that have a mass median aerodynamic diameter of 0.3 micrometers. While SARS-CoV-2 is around 0.1 microns in size, naked COVID-19 viruses in air are rare as they would be torn up nearly immediately, so they are almost always within aerosols. Typical respiratory aerosol range is around half a micron or so [0], and as the aerosol particle size gets smaller, so necessarily must be the amount of virus that is present.

Is this perfect filtration? No, but no one gets sick from a single virus they inhaled either, even with as communicable as the current Omicron variants are. There's a minimum infective dose and they help keep exposure under it.

[0] https://www.nature.com/articles/s43856-022-00103-w

The size of the single virus is a false metric here. There is a wide range of respiratory droplets containing virions. Those droplets can range from visible (way bigger than a mycobacterium ) to only large enough to hold one virion. The size distribution of those particles is the metric.

The basic problem is whether the data says masks don't work, or says that people aren't consistent enough in wearing masks.

I've seen it directly--one woman putting on a mask when I approached. The thing is she had been hiking near the back of the pack in a group that got together for the hike. She was at a far higher risk from being downwind of her group (this was not a family bubble) than of me being off to the side.

I can basically guarantee nobody there was experiencing any appreciable symptoms (10,000' up, miles from the cars--not something you're doing with any sort of respiratory infection) but most Covid spread is presymptomatic.

A solo hiker masking when someone approaches makes sense (and is what I did pre-vaccine), but not masking with your group but masking for a stranger? That's merely an illusion of safety and why masks "don't work".

There's also the problem that the Cochrane data included mostly studies of things other than Covid--when you go over their own data only looking at Covid you do see some benefit. Note, also, the pooling of masks and respirators--we already know masks do little against the Omicron variants. Respirators or don't bother.

Cochrane messed up badly in this case by looking at the wrong thing. I'm reminded of the BMJ study showing zero safety benefit from parachutes when jumping from an airplane.

Yes, the masks didn't work. Now everybody should know it. First of all, they were using paper mouth shields or adidas branded useless cloths, not masks. But even the dumb fcks using real n95 mask, i see people everywhere touching the mask from outside (where the viruses should be stopped if the mask works) and then touching everything else. And when coughing opening the mask and coughing inside the palm...

I would like to point out what “makes sense” to people rarely reflects the underlying fluid dynamics at the relevant scales. Couple that with a poor understanding of just how many particles one infected person emits and it’s clear masks as worn are very ineffective for the vast majority of people.

It also literally says "The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions."

It does, and that's true, but that doesn't contradict what OP wrote.

They found only mid-to-low quality evidence supporting the use of masks to prevent ILI. That evidence, for everything but the question of "n95 vs. other", showed an effect size statistically indistinguishable from zero.

You're essentially saying that the error bars on that effect size are big. They are. But they're still centered on zero.

The evidence they had was of such low quality that no solid conclusions could be made from it. What they found in the research may not reflect reality. They are explicit about this and stress the need for better research.

> "There is uncertainty about the effects of face masks. The low to moderate certainty of evidence means our confidence in the effect estimate is limited, and that the true effect may be different from the observed estimate of the effect...There is a need for large, well‐designed RCTs addressing the effectiveness of many of these interventions in multiple settings and populations, as well as the impact of adherence on effectiveness, especially in those most at risk of ARIs."

They admit that they were unclear about it and later were even more explicit.

"Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people's risk of contracting or spreading respiratory viruses."

The review is not able to "address the question" let alone conclude anything about the impact of mask wearing. The review is inconclusive.

> The evidence they had was of such low quality that no solid conclusions could be made from it.

OK, fine. If I grant this (I don't, but let's run with it...), it means the following is also true: we mandated something based on such low-quality evidence that no solid conclusions could be made from it.

Also, while this study was inconclusive as to whether masks help prevent covid, it doesn't mean that all studies are inconclusive.

For example, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8768005/ is "a detailed performance evaluation of the mask is studied from an engineering point of view," which aim to look at how the physics of N95 masks hold up against covid. What the physics shows is that N95 filtration helps block covid particles.

The success of the Nutrition Facts labeling does not get enough publicity.

Rather than outlawing certain ingredients, or creating some kind of health score which a product must be above, Nutrition Facts is a way for suppliers to attest to information about a product in way that is legally binding. If it isn't accurate the penalties are steep.

Consumers then have the information they need to vote with their wallets. Markets cannot function properly without symmetric information, and Nutrition Facts essentially creates a functioning market where one did not exist previously.

Any effort to regulate what's in food would probably be better spent expanding what must be in the Nutrition Facts label. I guess it's nice that we are finally getting around to banning artificial trans fats, but anyone who can read has been able to keep those out of their diet for years. The same can be said about the next bad ingredient, and the one after that.

This isn't without downsides, of course. The case of manufacturers adding allergens to food deliberately is alarming in its own way.

Surely, you aren't complaining of peanuts in peanut butter, so can you share an example, and why the allergen presumably shouldn't be there?

Apologies, I should have included a link. I am referring to https://apnews.com/article/sesame-allergies-label-b28f8eb3dc....

Mayhap that is overblown? I confess I have not followed it too heavily. Very thankful that I am not allergic to anything in my adult life.

I don't have any food allergies, either, so I admittedly don't pay much attention to allergens (or even listed ingredients most of the time). I'm genuinely curious, too.

I had the same questions and found this article: https://snacksafely.com/2016/06/kelloggs-unintended-conseque...

Basically, there were stricter measures put in place called HARPC. From my linked article:

> The new directives mandate that the “Top 8 allergens” identified by FALCPA (peanuts, tree nuts, milk, eggs, wheat, soy, fish, and crustacean shellfish) must either be ingredients of the product and identified as such, or the manufacturer must take extra care (and cost) to ensure that there is no cross-contact with them. There is no middle ground or “out” for the manufacturer, which is why we believe “May contain” type label advisories are heading for extinction. And that poses a problem, at least in the short term.

> companies, when faced with the added burden of instituting and documenting cross-contact prevention measures as dictated by HARPC, may instead choose to add trace amounts of the allergen to the product, as doing so makes the allergen an ingredient of the product and obviates the need for preventative cross-contact measures for that allergen.

> it means that manufacturers will either take stricter measures to prevent cross-contact or add a trace amount of the allergen and list it in the ingredient list, thus eliminating the ambiguity that currently plagues us all.

> A compromise that might have avoided the unintended consequences of companies like Kellogg’s adding traces of allergens to their products is to have offered them a third option: A mandatory “May contain” label advisory for any product made on shared equipment or in shared facilities that did not meet the FSMA threshold for cross-contact prevention. Such label advisories are voluntary today, rendering them ambiguous at best, but a definitively worded and located advisory statement included on all such products would have provided a way for manufacturers to meet the requirements of HARPC without resorting to the addition of allergens.

So I'd be surprised if it was happening on an ongoing basis, but I can definitely see why people would be irked.

The penalty for having an allergen present is steep, and the process of certifying that yes, you are in fact allergen free, is expensive and difficult, while the cost of adding an allergen into your process, for pretty much any foodstuff, is cheap, and the cost of slapping a "may contain peanuts" label on is cheaper.

I believe the original comment was complaining about the perverse incentive there.

And if may contain is not enough or allowed, may as well throw some amount of peanuts in and list it. And cover all the bases.

No, we are complaining about the way the government handles allergen labeling.

It used to be that companies could slap a "may contain [allergen]" label on things that didn't contain it but were produced in a factory where cross contamination was a possibility. Such labels are *widespread*.

I don't understand the government's incentive in trying to stop this--the actual result was when cross contamination was a possibility the companies reacted by deliberately adding the offending material.

The problem is that it's being looked at in a binary sense. Either it contains the offending material and poses a danger to those affected, or it doesn't and is safe. However, in the real world there's a third population--those who are sensitive to the offending ingredient but not dangerously so. Possibility of cross contamination? That is not going to be a deterrent to me as the worst case outcome is merely unpleasant. Does contain? I'm going to treat it with great skepticism.

I'll add, a family member has celiac which makes it so they can't eat gluten. Becoming certified "Gluten Free" requires a certification process that can be expensive and difficult. However many companies have realized they can label their product "Gluten Friendly" and get around the requirements. It is annoying.

Interestingly, there is some gaming of the main number everyone looks at on the nutrition facts chart: calories per serving.

All snacks aim to fall at or below a certain number the FDA (or some other agency) put out as being considered a snack. Planet Money did an episode on different M&M varieties having different total weights to account for their different calorie counts. So you get fewer by weight peanut butter M&Ms because they’re more calorie rich

This doesn't sound bad to me? At large, people eat a package of whatever snack they choose to buy. Also at large, people assume different packages of snacks should be roughly comparable for important metrics. And calorie count is probably up there for important metrics.

> There should be Nutrition Facts but for scientific trials.

No, there should be prison time for scientists who conduct unethical trials or publish fake results.

The public (and policy makers) place such immense trust in those people and what they publish that nothing less is even remotely adequate.

When someone puts arsenic in food, they go to prison – labeling the food with "contains arsenic" doesn't cut it.

Do this and watch science magically fix itself.

Revoke their license and titles to start with so they cannot operate anymore in the field

I would like to see Universities take the lead here. A bunch of high profile degree revocations would generate some waves at least.

As reasonable as that would be. I feel like it would just turn into people getting their degrees revoke claiming 'cancel culture' and becoming gurus with many mindless followers pushing loose weight quick schemes kind of thing. Because today if you face consequences, its no longer your fault, its everyone else trying to cancel you from doing the bad thing you are doing.

Which prison? As the article says:

> Ultimately, a lingering question is — as with paper mills — why so many suspect RCTs are being produced in the first place. Mol, from his experiences investigating the Egyptian studies, blames lack of oversight and superficial assessments that promote academics on the basis of their number of publications, as well as the lack of stringent checks from institutions and journals on bad practices.

A substantial part of what's happening here is that first-world countries with generally good cultures of research integrity are basing medical policy on studies done in countries where the system encourages researchers to cheat. British and US authorities can't put Egyptian or Chinese researchers in prison, can they?

There's no shortage of scientific fraud happening in the "first world" also. Dealing with those people would be a good start.

This is the correct answer. Typically if something is done at an ivy league, other US and UK universities follow. Perhaps the other countries would follow as well shortly after that, or there would just be a divide between 'real' research and 'not', similar to when many of my friends stop reading after the word 'Hindawi'.

Just to be clear, if you actually did this what you would see is stuff like Florida locking up every climatologist for doing "false science".

They would have to prove it to a jury.

I do agree it would be a major deterrent to doing such research, but as it stands you'll get fired for it anyway which is a pretty major deterrent.

It wouldn't be limited to Florida. Elsewhere, having any reasonable questions about the severity regarding the religion of Climate Change would get one jailed for blasphemy.

- Do this and watch science magically fix itself.

Imagine same approach in politics. If some politics put lies in their speeches to manipulate with people, we should call them liers and put them in prison. But somehow it doesn't happen. Looks like society prefer conformity over responsibility.

Why would the statisticians be anonymous? I'm aware of at least a couple cases in which an independent set of statisticians were provided the data from a clinical trial specifically for a re-analysis. In one case, they showed some pretty concerning inconsistencies and the other confirmed no effect on the primary analysis, but suggested some sub-populations that might have shown an effect if a future study was properly powered. That follow-up clinical trial was just published showing pretty remarkable effect in the sub-population. I don't think there's reason to believe either independent analysis was anything other than independent.

There is already a mechanism for companies to submit questions to the FDA prior to clinical trial initiation. I'm not in these conversations, but I know the type of questions can be things like: would you accept this endpoint as a proxy for this indication, would you be satisfied with the effect size we expect, and are there other safety concerns you would expect us to evaluate other than those in our current plan. I assume EMA and other regulatory bodies have a similar process, but I'm not positive.

Disclaimer: I work in pharma, but pre-clinically. I am not involved in these clinical or regulatory issues.

So junior scientists can be hired without them being concerned for future career prospects.

Yes, I have said this as "The UX of [medical] study papers is terrible". Some people do not agree, they think that it should not be made easier to understand, that non-experts cannot really understand medical studies, so they should not be more approachable. I think that's dead wrong.

Im struggling to see the difference between this and the current FDA process, and think it is 90% the same.

Drugs have "prescribing information", referred to in industry as "labeling", which follows a consistent format containing safety, trial results, side effects, and mechanism of action.[1] I recommend people read them for drugs they take.

Third party analysis is the main difference here. In the current state, firms run the analysis for FDA review using standard practices, and must explain and get approval for any unconventional methods

>There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).

Labeling contains many relevant numbers. Trial sizes, how many per arm, what was measured, and and final results. Maybe there could be some squishy qualitative summary, but that seems more risky. I would rather know that 1 out of 20 patients died than it got a "2" on the safety scale.

>Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).

Labeling contains drug specific information.

There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.

Drug labeling requires pre-application and and a standard 12 month review period by the FDA prior approval

The FDA provides advice on structuring trials and acceptable design, size, power, endpoints. Firms do this by scheduling calls with FDA staticians and experts. [2]

And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.

Maybe there is a difference here. I'm not sure what you mean by punishment? In the current system, The FDA can use the label as "punishment". The FDA may require addition "black box warnings" for drugs that are found to have serious side effects (e.g. high chance of death). They can also pull the label entirely, meaning the drug can not be sold.

This is basically how it works for medical device labeling in the EU. There are several "notified bodies" [3] which are private agencies to review the safety and efficacy. The firm then takes their mark of approval to the government agency.

https://www.accessdata.fda.gov/drugsatfda_docs/label/2014/20... https://www.fda.gov/media/72253/download https://climedo.de/en/blog/list-of-mdr-certified-notified-bo...

The async thing means papers can be published without waiting for a gov agency to do its review. FDA takes the opposite approach.

Generally, reading a paper is much less risky to a person's health than taking a drug so the differences in review process add up meaningfully.

Reply on news.ycombinator.com