Comment by MauranKilom | Hacker News Reader

Comment by MauranKilom | Hacker News Reader

> 7/ For sure, there are many studies that study variations of surgical methods. That is, they try to find out whether an incision from the left is better or from the right.

> But very few studies try to find out whether the incision does anything at all for the patient. Shocking, but that’s the truth.

Assuming that those studies do not always conclude "neither is better than the other", this implies one or both of these:

1. At least one of the two options is better than the placebo.

2. At least one of the two options is worse than the placebo.

In other words, finding differences in effectiveness of different intervention approaches must mean that either some of these interventions are better than placebo, or that all (except maybe the best one) are measurably worse than performing no surgery at all.

> "the severity of your symptoms does not change the effectiveness of the operation"

I hate the word "effectiveness" because it can mean anything. Effectiveness at what? Measured how? Surely the severity of my symptoms ("my toe hurts" vs "half my leg has been shot off") plays a role in determining whether amputation is an "effective" remedy? Or you could argue that it is equally effective at "removing" the symptom in both cases. But then killing the patient would be peak effectiveness - no more pain at all, guaranteed! It's just pointless to discuss such a thesis - if you talk about "effectiveness", your statements very quickly become unfalsifiable.

> In other words, finding differences in effectiveness of different intervention approaches must mean that either some of these interventions are better than placebo, or that all (except maybe the best one) are measurably worse than performing no surgery at all.

That doesn't tell you anything useful about them vs. placebo though? And "all (except maybe the best one)" seems to be entirely gut instinct not logic. You don't need to even try the surgeries to know that either all, some, or none will be better than placebo.

If both types of surgery perform the same, they could both be better, both be worse than, or both be the same as placebo. If one type is better than the other then maybe they're both better than placebo in different amounts, maybe both worse in different amounts, maybe one better the other worse than placebo, maybe one is same as placebo and other better, or one the same and other worse.

Knowing that two surgical approaches give different results gives no hint as to where placebo would perform in comparison, just that it can't be the same as both of them. It can still be better than both, or worse than both, with no reason to suspect one more than the other without other priors or actually testing vs. placebo.

There is always the third possibility: random statistical fluctuations make one option seem better than the other when this is not true.

Which breaks down to two more possibilities in relation to placebos:

1. Both are as effective as placebo. 2. Both are equally effective and better than placebo. 4. The seemingly worse option is actually better than placebo (but statistical fluctuations masked this).

Given that there are very few studies and that publication bias is a thing, these possibilities are conceivably more likely than the p-value would lead one to believe.

The benefit of placebo testing in the first place is that you end up with a simpler possibility matrix (the neither is better than the other outcome also breaks down into a whole slew of cases).

Effectiveness as compared to a placebo.

Of course, patients get better after surgeries. But the rate at which they get better is not much better than if surgery wasn't performed (or a placebo surgery was done).

Reply on news.ycombinator.com