Hacker News Reader

> If you did this in a randomised controlled trial, you might recruit 1000 patients, and randomise 500 to this new drug and 500 to placebo. If you ran the trial for a year, you might find 1% of the placebo arm died, and only 0.5% in the control arm. This demonstrates a risk reduction of 0.5%, or a NNT of 200.

That's because the only thing you've learned during the above trial is that it halves the death rate for one year. You have no idea what happens after that year. It could get better, worse, or the adverse effects would be worse than the cure.

Your assumption that that drug cuts the death rate beyond that single year is an assumption, not a fact.

> An NNT, by conventional teaching, is underwhelming.

As it should be. That's the only thing you can conclude from the trial.

> Is this the right way to think about it? ABSOLUTELY NOT! The high NNT is a consequence of a low event rate, NOT an ineffective therapy.

Yes. The absolute proven benefit is miniscule.

> In preventative medicine, where everyone will EVENTUALLY die of something (and 25% the thing you are looking at), it is the RELATIVE risk reduction (50% here, not 0.5%) that matters.

But you've only had a 1 year trial. Drugs often have long term adverse effects that dwarf the benefit.

As an example, when steroids were first discovered they were viewed as miracle drugs and given in massive doses. It didn't take long to understand the adverse effects were far worse than the symptoms it was treating. Today we are far more careful with steroids.

Similarly with Aspirin, or any number of very useful drugs.

So the question is what gives you the confidence to make any claims beyond what the trial showed?

You've probably thought about this more than I have, but your "take down" of NNT doesn't work for me. Let's use your example. We treat 500 people and observe another 500. 5 of the untreated die, and 2.5 of the treated die (I don't know how either). We observe that the NNT is 200.

And we this is great! If a drug is inexpensive, has minor but not severe side effects, and saves 1/200 people from dying within a year you should absolutely be giving it to everybody. It's a home run. The problem isn't the NNT, it's the expectation of what constitutes a a high or low number.

By contrast, consider another study. We treat 500,000,000 people, and observe another 500,000,000. We observe a 50% relative risk reduction among the treated. The drug is inexpensive, and has minor but not severe side effects. Should we create everyone?

I have no idea. It has to depend on how much risk we started with. If the mortality rate is 1% per year among the untreated, then almost definitely yes. But if we observed 2 deaths in the untreated and only 1 from the treated, probably there is a better area to spend our efforts, even if we somehow knew the life we saved just noise.

So yes, the reduction in risk matters, but only in concert with a concept of the baseline risk. By itself, it's not very helpful, but in concert with other data it's great. NNT is a way of expressing this in a single number. There is no apriori reason to believe that 200 is high or low. It may not be perfect, but it's better than just a bare relative risk percentage.

But how about if you project it into health economics? If you use it to calculate something like QALY cost?