Generally you should assume they are measuring symptoms of infections that go beyond the upper respiratory tract. This is because the vaccines provide little protection against an upper respiratory tract infection but instead are very effective at preventing it from progressing further into a more severe infection. This is referred to as non-sterilizing immunity: it is probably the most important concept for everyone to understand about the vaccines.
I am seeing a lot of comments here claim that vaccines are 100% effective at preventing death or severe infection where severe is generally defined as hospitalization. Nothing is 100% effective, and particularly not vaccines for those whose immune system does not respond properly. Death in particular is generally not an endpoint that can be compared in a statistically significant way in these studies. [1]
I will still be getting the first imperfect vaccine I am allowed to at the first chance I get. When I needed to get an appointment for my Mom I started writing a user script to help with that. [2]
[1] https://dalewharrison.substack.com/p/vaccine-boosterism
[2] https://gist.github.com/gregwebs/265e0ef6b1a3051377cfc4d66ac...
They prevent symptomatic covid infection which includes respiratory symptoms.
- Take two identical[1] populations of people
- Wait for some people (100 for easy math here) in the unvaccinated group to get the disease. Probably measured by a PCR test indicating presence of the virus and ignoring symptoms since they say "preventing disease" and not severe symptoms, hospitalization, death etc.
- Count how many folks have the disease in the vaccinated group[2]. In the "91%" case we would expect 9 people to test positive, meaning 91 of 100 people we would expected to get the disease did not.
Is this what they mean in this?
[1] Doing your best to create identical groups by controlling for population differences such as gender, age, and other known risk factors,
[2] We need also to control for time in the study etc.
Same, and it gets less likely if you are in a group of vaccinated people since the chances of being exposed to someone who is sick goes down. Herd immunity at that point.
I suspect most of the public doesn't even know what 91% means. Combine that with being bad at evaluating risk in general. It mostly represents a 10x improvement in cases per capita per time, but I think telling folks that their risk went from 1 in 10,000 to 1 in 100,000 (or whatever) probably doesn't mean a lot. Folks just want "safe" or "not-safe" :(
> Is this what they mean in this?
Yes, that's what they mean. And even for the vaccines that are ~60-70% effective, all of them have so far been 100% effective in preventing hospitalization and death. It's not binary, the vaccines don't make you completely immune to this virus but greatly reduce the impact it has.
I thought this video explained it well: https://www.youtube.com/watch?v=K3odScka55A
If anyone wants to look at the raw phase 3 numbers:
Pfizer numbers: 43,448 participants received injections (21,720 the vaccine and 21,728 the placebo); 8 cases of COVID-19 in the vaccinated group and 162 in the placebo group, 9 were severe (8 of those in the placebo group).
Pfizer study: https://www.nejm.org/doi/full/10.1056/NEJMoa2034577?query=fe...
Moderna numbers: 30,420 participants, evenly split with 15,210 in each group, over 96% got both injections. There were 185 symptomatic cases in the placebo group and 11 in the vaccinated group. 30 participants had severe cases including one causing death: all were in the placebo group.
Moderna study: https://www.nejm.org/doi/full/10.1056/nejmoa2035389
That might make the vaccine even more effective than the study shows if the stats don’t back out risk adjustment (which they probably cannot do).
But note that wearing a mask is still required. At a technical level, you may still be able to spread the disease, even if you're fully vaccinated. And the vaccines aren't 100% effective, so even the minimal self protection you get from a mask is better than nothing.
More importantly, I believe, is sociological: once there are a lot of people in public without masks, even if it were safe, a lot of people would take it as an excuse not to wear one. That leaves both them and others open to getting sick. The mask is a minimal inconvenience, and should be continued.
If you want want to skip the mask among people you know to be vaccinated, in private, that's about as safe as anything ever gets in life. Even small, private gatherings with a cluster of unvaccinated people (i.e. people who all live together) are reasonably safe without masks for those who are vaccinated.
I'll do it for many more months, because the sociology aspect is valid.
You'll need to read the actual trial protocol for a specific vaccine to find out just what they were measuring, and how they defined e.g. "serious" vs. "non-serious" cases.
They basically split people at a certain area into two groups, give one the real thing, another one placebo, track infections, compare the results.
Comparing vaccines by effectiveness is meaningless because they're not all tested at the same place at the same time. So J&J has lower effectiveness, but it was calculated at the peak of the outbreak, when each participant interacted with Covid more frequently in their everyday lives.
For the comparisons to become meaningful, it'd need to be re-done at the same place at the same urban area, with 10s of thousands of participants for each vaccine. You'd also need to have approximately similar age spread, plus similar population distribution within an urban area. Since it's all volunteer-based, this kind of study won't happen.
In conclusion: take any that's available to you.
I'm not sure I can envision a reasonable way to embargo the numbers.
Probably more considerations, but the above should get you started thinking of them.
J&J was 85% effective at preventing severe reactions and hospitalizations and 100% effective at preventing death. The 72%, in the US, was for mild symptoms.
There's nothing wrong with saying X is much better than Y, Y is much better than nothing.
I get from a public health standpoint you may want to blur the difference so people take the first available dose because there is a huge value to herd immunity, and a society with 100% J&J shots is going to be safer - probably even for the vaccinated given herd immunity - than 20% with mRNA shots. But I won't pretend they are equal while doing so.
Meanwhile, the 72% vs. 66% was when the world had variants not seen in the US. They're here now. I think it more likely that the vaccine has less efficacy against the variants than the vaccines have special "efficacy +" modes that are geoblocked.
Obviously if you can get any vaccine, do so! But to claim that we can't possibly compare vaccines is silly. Yes the trials have differences, but they're not so different as to make us totally ignorant. We can use the trial data plus knowledge of the mechanism of action to make solid bets on relative effectiveness.
Especially important since we have kids.
The whole point of a vaccine is to prime your immune system, it does not provide a magical shield to your body. If you get exposed to COVID-19 after being vaxxed, you will still have COVID-19 in your system for some amount of time. It will just (hopefully) get killed very quickly when the memory B/T cells (that were created by your body when you had an immune response to the vaccine) ramp up production.
I was paid to work specifically on COVID, so was monitoring the vaccine data very closely. It was *very* promising, but there were a few things that puzzled me in where comparing results in placebo/control group vs vaccine groups, for both Pfizer and Moderna.
So I would check near daily for new results.
And then after Christmas break, when I went to check, found out both control groups had been effectively destroyed. (https://www.wsj.com/livecoverage/covid-2020-12-17/card/Pc6LV...)
I'm not kidding, there are no longer any control/placebo groups for these vaccines. These were studies that were approved to run for 2 years!
I had to sit through hours of boring mandatory training from NIH on importance of control groups. In these cases they just threw that away.
It reaks to high heaven to me.
(Note: I've seen nothing to indicate these vaccines are dangerous. Effective? Looks very likely, at least in the short run. Long run effective? HIGHLY skeptical. Total contribution to ending the pandemic from the vaccines? I'm highly skeptical that it will actually be high, when an honest accounting comes out). Would I personally take the vaccine had I not had COVID already? Yes. Though not if I were a kid or teenager (very little risk from COVID).
If you are in medical research, please don't look to these people as role models. Please do things openly, on git, and don't sweep uncomfortable truths under the rug.
The numbers do not come close to supporting any ethical reason for doing this.
For Covid, there's an additional problem with antibody tests being relatively abundant, and would allow the participants to unblind themselves anyway.
Actual Study Start Date: July 27, 2020
Estimated Primary Completion Date: October 27, 2022
I thought there was going to be a two year control group because that was the plan published on ClinicalTrials.gov run by NIH.
> The moment somebody in a trial could get vaccinated via some other channel, they'd drop out and get unblinded
If this is such obvious common knowledge why wasn't it in the plan? Why isn't it in the training?
I am 100% open to fundamentally and drastically changing the way we test medicines and vaccines et cetera, but that's very different than just making decisions willy nilly that just so happen to align 100% with shareholder interests.
(But fair enough; in the case that no vaccine got an EUA, the control group would have lasted for two years.)
All that the protocol change to give the control group the Moderna vaccine at that point did was to let them at least continue with the observational open-label phase B of the study.
Your suggestion that this a sinister plot to hide the long-term inefficacy of their vaccines is absurd. A Covid vaccine that was highly effective for only a couple of years would be a goldmine in the long term, and would sell just as well right now.
Exactly. And some would. But most would not. Just like every other randomized control study. It boggles the mind to say "let's tell everyone they got the placebo, because some people might leave". That does not follow logic.
> plot to hide the long-term inefficacy of their vaccines is absurd.
What part of that is absurd?
At the time the control groups were destroyed, there was no evidence that the vaccines were saving lives (https://www.fda.gov/media/144434/download — 7 deaths in placebo group; 7 in control group). Now, if you got tens of billions of dollars based on those early results, wouldn't you have *strong* incentives to shut things down, and not wait for the long-term verdict? What would you have to gain if 2 years in, the placebo group continued to have just as few deaths as the vaccine group?
Are you saying that the idea that humans respond to incentives is absurd?
How could that possibly be true? Basically everyone who isn't a rabid anti-vaxxer is trying to get a Covid shot as soon as possible. And pretty obviously none of the participants in a vaccination study is going to be an anti-vaxxer.
> 7 deaths in placebo group; 7 in control group
That's not true. Why in the world would you fib about something that is this easy to verify? There were 3 deaths in the treatment group, 4 in the control. But more to the point, there were no deaths from Covid in either group. It's not that "7 people died of Covid in the treatment group" which you were clearly trying to imply.
Now, why were there no deaths from Covid? Because the size of the trial was set such that they could reasonably expect to find an effect for preventing symptomatic disease. That's why it was the primary endpoint.
Why couldn't they run a trial where Covid deaths were the endpoint? Because the IFR of Covid is "only" 1%. To run a trial that could reliably show an effect on the number of deaths, you'd be looking at 100x the participants. I'm sure you're not seriously suggesting running a trial of 3M people.
> Are you saying that the idea that humans respond to incentives is absurd?
No. I think your suggestions on what Moderna's incentives are absurd.
Think about it for a bit: the early results showed no change in the number of deaths. You're suggesting that if they waited and... ummm.. showed no change in the number of deaths, it'd somehow lose them tens of billions. I can't help but notice that the before and after are the same there: "not showing a change in the number of deaths".
But also, that's 1.5 years from now. There's a lot of vaccines to be sold right now, and that's what their incentives would be focused on. Not on hypothetical effects that their trial was not designed to surface in the first place.
And again, the foundation for this theory is that the trial participants would have agreed to not get a working vaccination for two years, even if on placebo. You've not presented any proof for this except claims about "NIH training".
Sorry, that was a typo from memory: not 7 and 7 but 4 and 4. Conclusion is unchanged. From the FDA link I provided, Table 19. Here's a screenshot with the relevant line https://twitter.com/breckyunits/status/1348080756921303041/p...
> which you were clearly trying to imply.
I absolutely was not trying to imply these were COVID deaths. I'm trying to imply that the wholistic expected value of the COVID vaccine needs to be considered. In my personal circle of ~2,000 family and friends, I have lost 4 people in the past ~12 months from Non-Covid (35, 50, 65, 62), for a total of about ~100 healthspan years lost, versus 1 person from COVID (with a healthspan left of ~1). I know >100 people confirmed COVID. One cannot make decisions in isolation.
> Now, why were there no deaths from Covid?...that their trial was not designed to surface in the first place.
Because they cut the study short! In the clinical study protocol (https://www.modernatx.com/sites/default/files/mRNA-1273-P301...) they specifically state "To evaluate [Vaccine efficacy] to prevent death caused by COVID-19" as an objective!
"The study is designed to primarily evaluate the clinical efficacy and safety of mRNA-1273 to prevent COVID-19 for up to 2 years after the second dose of mRNA-1273...Participants are considered to have completed the study if they complete the final visit at Day 759 (Month 25), 24 months following the last dose of IP."
> the foundation for this theory is that the trial participants would have agreed to not get a working vaccination for two years, even if on placebo.
I can't speak to what the experience of people in this trial was like, as I was not a participant, but I have never been involved with a control group where they have told you that they would tell your your placebo status not even halfway into the study.
> I think your suggestions on what Moderna's incentives are absurd.
Moderna the corporation has no incentive to keep the control group going because they've already closed the sale so if the long run efficacy turned out to be not as strong, they potentially could be exposed to downside risk. I am not against the vaccine or the rollout. I'm against the destruction of the control group.
Is it? I don't remember reading about that in NIH training on control groups. I'm pretty sure when your plan is to distribute 1 billion vaccines, holding out 0.000015 for a placebo control group is a sensible "ethical" tradeoff. Especially if, I don't know, everyone *volunteered* and agreed to those terms when signing up! And especially if, I don't know, all the data they had to date showed no increase in death rate?
Also HN may shadow ban you.
I don't think HN shadow bans me. My goal on these sites is high variance. High upvotes or high downvotes. In my experience that's the goal—either create a lot of value for people or learn something.
I also have accounts on not my real names for more pleasant, positive, uplifting content, but I save the controversial stuff for my real name accounts.
All the vaccines so far are 100% effective against "preventing severe disease"
The last one (non-symptomatic), we don't really know, I don't think any studies have been done which regularly test everyone, at scale. Someone correct me if I'm wrong.
https://www.nejm.org/doi/full/10.1056/NEJMc2102153
https://www.nejm.org/doi/full/10.1056/NEJMc2101927
2 weeks after the second Pfizer dose, one of the medical centers had a positive test rate of 0.05% (not a typo, 0.05 percent).
I'm a little confused by the percentage though. Looking at the 2nd study, and ignoring 1-7 days after the second dose, there's still 8+7 people who got infected a week after the 2nd dose, out of ~5000 tested. How do they get to the 0.05% number? Also, if I understand correctly, the table says there's around 16,000 eligible, but only 4000-5000 were tested? Am I missing something?
I believe Pfizer has even started doing 3rd shot boosters for very early phase 1-2 people from last year.
Efficacy in vaccines == people who were prevented from being confirmed COVID positive (symptoms sufficient to prompt a test, leading to a positive result)
Therefore, a 91% effective metric for vaccines means that 91% of those who receive that vaccine are expected to NOT contract COVID 'at all'. Thing is though, many vaccines broadly do not prevent disease but shift the severity upon contraction.
The metric that is actually useful for informing the public and policy is the answer to the question of 'does this prevent hospitalization, severe disease and death?'. Broadly speaking most approved vaccines globally have near 100% effectiveness in this.
The efficacy is the 1-relative risk of the primary endpoint.
But I was just reading about another study of medical personnel who were taking periodic COVID tests so as to catch asymptomatic transmission, and I believe this study (or maybe merely the media about it) also used "effectiveness".
It does not mean 10% of people end up with no protection at all, as was my first thought.