Hacker News Reader

Comment by diroussel | Hacker News Reader

It's hard to analyse the data from treatments, as treatments don't have a control group. Trials are designed with control groups, placebos and double-blind process to gather data and then analyse it.

We may have loads of data from real people being treated, but good luck trying to analyse it in a scientific way.

We need effective vaccines, and we need to be able to trust them. Just treating people with a vaccine doesn't always give that trust and confidence.

(I hope I got my medical terminology right, I'm not a clinician, but have been close to covid-19 related medical research)

View on HN

Depending on the phase of clinical trials, they don't need to be double blind or controlled.

You can absolutely analyze real world mass vaccination data in a scientific way. Just treat pre-vaccination and post-vaccination as two different patients. The numbers are so large that other factors (environmental, genetics, etc) cancel out. So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation.

(Full disclosure: my family used to own a company that ran clinical trials for pharmaceutical companies)

> You can absolutely analyze real world mass vaccination data in a scientific way. Just treat pre-vaccination and post-vaccination as two different patients.

Yes, you can do this. No, it's not the same.

With this kind of uncontrolled, longitudinal study, yes, some things are quasi-controlled: genetics, possibly lifestyle, other drugs, etc. Anything that plausibly doesn't change (much) in a single person from time A to time B.

Some things you can't control for, but still matter: changes in behavior due to the event itself. Placebo effect. Sample bias (e.g. the patients receiving treatment X were selected to receive treatment X because it was felt that they would benefit from it. This is subtle, but can really mislead. In the example of vaccination, imagine that the people most likely to vaccinate their young kids are also the ones most likely to keep their kids isolated at home, in a protective bubble...)

> So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation.

The key part of that is the last three words: worth further investigation. To get the final answer, you still need a controlled experiment.

Different prevalences of variants over time and different rates of infection over time would complicate such a time-based control.

It's much, much harder to understand the data if you don't have a control group. But once you have an effective treatment, you no longer have a control group if you're at all ethical. You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical.

So the moment we knew the vaccines worked, the science got harder. Well, that point also save an enormous amount of lives, so I don't think this is something to complain about. It's still possible to do a lot of good science now, even randomized controlled trials. They just can't compare to unvaccinated groups, but you can still do stuff like compare double-vaccinated with triple vaccinated.

A lot of data now shows that double vaccinated are infected with Omikron more often than unvaccinated. I do not know what causes this or what it means, but imagine for a moment there were no unvaccinated left: Your vaccine could be arbitrarily harmful with a new variant and you would not be able to tell.

That is simply wrong.

There is one study that listed negative effectiveness against Omicron for the vaccines. That study did not control for any confounding factors, it simply was not designed to measure this particular thing, it was focused on a different question.

There are some strong fundamental reasons why we would not expect the vaccine to be harmful with future variants. And even then we would still be able to detect if it was.

It is not one study, national data from Denmark, UK, Germany and Canada shows that double vaccinated are infected at a higher rate. I am not jumping to conclusions that this means efficacy is negative, because that would require a controlled trial with an unvaccinated group. Do you catch my drift?

Please show the data that claims that double vaccinated are more likely to be infected than unvaccinated people by Omicron. And raw infection counts are not an answer here as the vaccinated and unvaccinated people are clearly not matched populations you can compare without adjustments.

> And raw infection counts are not an answer here as the vaccinated and unvaccinated people are clearly not matched populations you can compare...

That's the point! That's why we need controlled trials. With an unvaccinated group. You know, the thing that you say we can't have, because it's unethical.

> ...without adjustments.

As usual, there's an XKCD for that:

https://xkcd.com/2560/

There’s no way of running a controlled trial of this sort when 80% of the population have already been vaccinated. You can design better or larger observational studies though.

> But once you have an effective treatment, you no longer have a control group if you're at all ethical. You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical.

You could easily find millions of potential study participants who don’t want the vaccine. The study wouldn’t be double-blind, but it would be single-blind and better than what they chose to do.

It's astounding how rare this rather obvious course of action has been in the global conversation.

> you no longer have a control group if you're at all ethical.

> You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical.

I see this position put out there a lot and generally go unchallenged. For the record some people think the opposite: that it was unethical to unblind the placebo group so early. At the time you could estimate that continuing the placebo group might lead to ~60 unnecessary deaths from Covid-19 and a few hundred more serious cases. That is in the lucky scenario where the early success held. But in all scenarios, in return for a few dozen volunteers risking their lives, billions of people would benefit from randomized, placebo-controlled trial data which the NIH itself calls the "gold standard" as to "whether or not a treatment is safe and effective".

IMO the harms we have gone through from flying in the dark (people underhyping/overhyping the vaccines), multiplied by the number of people involved, make this a case where unblinding was unethical. It is very easy to imagine that far more life was lost in the general population from being in the dark than was saved by unblinding the volunteer study population.

(There's also the issue of whether or not it would have been practical to keep the participants blinded. I think that's a challenging topic of its own orthogonal to the ethical question.)

the covid vaccine trials had no control groups either

They were all doubly blinded placebo controlled trials…

Are you talking about how they unblinded the study after efficacy results started coming in?

yes i am talking about how they got rid of the control groups once it appeared to be working

Reply on news.ycombinator.com