Hacker News Reader

Comment by jseliger | Hacker News Reader

RCTs are fine but the obsession with them is overwrought and counterproductive. My own drum to beat on this is regarding clinical trials for fatal diagnoses like cancer: https://jakeseliger.com/2024/01/29/the-dead-and-dying-at-the.... We have Kaplan-Meier curves for fatal diagnoses. We know what happens (the tumors grow and metastasize. One doesn't need elaborate phase 3 RCTs to figure out if there's a good shot that a treatment is working; one can see it in tumor response and comparison to known KMCs. The existing system raises costs and causes people to die while waiting a decade or more for exciting treatments: https://atelfo.github.io/2023/12/23/biopharma-from-janssen-t...

Moderna's mRNA-4157 is a current example of this: https://jakeseliger.com/2024/04/12/moderna-mrna-4157-v90-new..., although it may be held up by lack of manufacturing capacity as well.

View on HN

> RCTs are fine but the obsession with them is overwrought and counterproductive. My own drum to beat on this is regarding clinical trials for fatal diagnoses like cancer:

RCTs for mental health conditions are a completely different situation. The short-term placebo response rate for cancers is not high (obviously) though the influence of unblinded trial operators making subjective analyses can be a problem.

Many mental health conditions, on the other hand, have unbelievably high placebo response rates over the duration of a short trial. The magnitude of the placebo response is almost hard to believe in certain studies.

The placebo effect can be a problem for approving new drugs as some times the placebo group improved so much that there isn’t much room left for the active drug to improve beyond that. This is a problem of study design and rating systems that is difficult to solve.

Unfortunately, some study operators use this fact to their advantage by omitting placebo group. Without a placebo group, it’s not obvious that the drug is actually doing anything better than placebo, of course.

> * Many mental health conditions, on the other hand, have unbelievably high placebo response rates over the duration of a short trial*

Probably how faith healing works.

If someone could explain how "the placebo effect" is not just "scientifically proven faith healing" I'd love to hear it.

Placebos doesn't heal. They just seem to relief some symptoms typically self reported, like pain, but the underlying cause is still there.

The tricky thing about that (which isn't false, per se) in the context of mental health is that "relieve some self reported symptoms" can actually be sufficient treatment. As with many sorts of pain, if the patient feels better, _they are better_ in a meaningful sense. Whether it's "real" is sort of beside the point, especially if the problem is that they are (for example) too miserable to do normal life things that would stop them being miserable and the placebo is sufficient for them to feel as if perhaps they could.

Also, relief of emotional discomfort can help the patient adopt more behaviors that are associated with positive changes in mental health, such as exercise and pursuing social engagements.

Not necessarily. The placebo effect is the kind of thing where you might have trouble at first telling if your symptoms are improving or if you’re just having a “good day” or an “easy week”, and that confusion can even last a month or two during which you’re over-observing your internal state and feeling hopeful that “maybe this is what getting better feels like”. But in the long term you often figure it out.

I had a placebo effect recently when switching ADHD medication to get around the shortages. For a couple months I thought there was a chance my new meds might actually be better, they definitely felt different (and still do). But six months in it’s clear to me that I’m struggling with productivity more than I was before I switched (though less than when I was off meds).

I’m just one guy, but I’d guess this is why doctors don’t just prescribe placebos all the time as actual therapies (well, that and they’d lose credibility which would then destroy any remaining placebo effect).

Warts are overall very responsive to placebos.

My M.D. father, family practice in the army, later a pathologist, would do what he had learned from other doctors: Put some dye in toothpaste, put it on the wart(s), bandage it, talk about what a miracle cure it was etc. He said it worked the few times he tried it.

FYI, food dyes are not inherently inert, and are capable of having antifungal, antiviral, and/or antioxidant effects. Quick examples:

https://link.springer.com/article/10.1007/s10068-011-0002-0

https://www.purdue.edu/newsroom/releases/2020/Q3/new-approac...

The pigments fungi produce are so essential to their survival, they can’t defend against pathogens when they are gene edited to stop producing them.

Toothpaste is also far from an inactive substance, there are definitely plausible mechanisms at play with the toothpaste and dye mix that could help suppress/resolve a wart. It would be worth a study, though I’m not sure what one would use to try and achieve a truly inert placebo for comparison without first figuring out what doesn’t work.

Nothing to do with placebo, it's the covering that did the trick, not the talk about miracle cure.

(I discovered this myself when I was a kid, any proper airtight cover is likely to get rid of it, YMMV)

Stress negatively affects the outcome of patients, why couldn't the opposite be true?

Mental health conditions are typically defined by a collection of symptoms though, aren't they? I am not suggesting there is no underlying cause, but our ability to detect and quantify that cause is lacking, so defining disorders based on a collection of symptoms is what we are mostly left with in many cases.

It varies a lot. For many conditions, like cancers, the placebo effect is basically random noise (cancers sometimes just reduce or go away on their own, regardless of treatment or even of the appearance of treatment ; we don't know the natural rate of this, because it's unethical to not treat people who you know have cancer). In addition, in non-blind trials, there is a "placebo effect" that amounts to mistakes or lies by those involved in the research in favor of a positive outcome. This is not a real effect in patients at all, just an artifact in the reported data.

Then, for conditions linked to our psyche, including pure psychological conditions but also things like pain, blood pressure, heart rate, nausea, and some others - the placebo effect is more real, but usually temporary. Some people who have been living in some amount of despair at their condition experience a positive surge of hope once treatment starts, and they can ignore the pain, or feel some push to get out of their depression, or calm their anxiety which was exacerbating, say, the high blood pressure etc. This effect almost always tapers off if the treatment is not doing anything more fundamental.

Coupled with the fact that we don't understand how psychological disorders work at the chemical level at all, especially in relation to the conscious mind and interventions on that (e.g. therapy, but also various religious practices), this means it's very hard to account for this without a double-blind RCT.

The problem as I see it is that all medicine is fundamentally trying to find effective scaffolding on the human body, triggering it's own ability to heal. A surgeon can't repair a corpse. What causes the body to heal itself more effectively? I would think this is brutally difficult to study, since it's all subjective.

I think part of medicine is that, but part of medicine is trying to keep you alive despite your body. Or maybe what I'm arguing here is the "body healing itself". But for example, if you have autoimmune disease, or allergies, you want the body to slow down and take it easy because it's harming itself.

sure. How effective is faith/placebo at curing those conditions I wonder?

> they can ignore the pain,

Just to note that on pain specifically, belief that one has been administered a drug can cause the body to synthesise painkillers. This has been most rigorously demonstrated by the fact that these painkilling effects are suppressed by naloxone (an opioid antagonist).

Fascinating

I'm fairly convinced this is not much different than the way that your body will prime itself if it knows you have an alarm going off at a given time. I don't have the paper anymore, but it was shown that your hormone profile will change with just the knowledge of an alarm. Similar results have been found for other drugs and using cues to the body so that it will prime them itself.

I'd love to read more on how this links to the powers of ritual and general routines. Specifically, if I'm not misremembering, it isn't just "belief that one has been administered a drug", but it has to be a drug that you have had before. Or that you have seen work on someone else. Just taking sugar pills does nothing. Taking sugar pills that you thought were the aspirin pills you took last time you were sick can cause the body to react.

> I'd love to read more on how this links to the powers of ritual and general routines. Specifically, if I'm not misremembering, it isn't just "belief that one has been administered a drug", but it has to be a drug that you have had before. Or that you have seen work on someone else. Just taking sugar pills does nothing. Taking sugar pills that you thought were the aspirin pills you took last time you were sick can cause the body to react.

Not always, you can see effects from open placebos, where you tell the participant that they're getting a placebo but placebos have been clinically proven to reduce pain.

It even seems to work when you warn participants in the consent docs that you may lie to them (authorised deception).

I agree that it's probably a broader effect than just sugar pills, it should probably be called expectancy effects.

Benedetti et al have done a load of work on recovering surgical patients that suggest that many, many drugs (including valium) mostly work based on these kinds of effects. Its a fascinating field (and what I did my PhD on).

The placebo effect is not a single thing. That is, there are ways of amplifying the effect and minimizing the effect. For maximum effect, treatments ought to be designed to take advantage of the placebo effect to the extent possible. That’s because placebos have low side effects and are often very effective. However, this creates some challenges — how do you test which placebo effect works best? What do you use as a placebo? It’s not that hard, really — just use a placebo with less of a placebo effect.

Niacin was used as the placebo for Timothy Leary’s Good Friday experiment [1], where he randomly dosed catholic monks on psilocybin. Unlike a sugar pill, Niacin creates some facial flushing — so you do feel something. But it would be very clear eventually that you didn’t get the psilocybin. But that doesn’t negate the findings of the experiment.

[1] https://en.wikipedia.org/wiki/Marsh_Chapel_Experiment

Sure, for those conditions that have subjective components (e.g. pain, mood) or where there is more or less direct conscious control of the condition (e.g. heart rate, BP), you can vary the strength of such effects.

But in many other conditions, you can't, because that kind of placebo effect is just noise. For example, you can't vary the effectiveness of placebo effects in antibiotics studies (though you may be able to reduce certain side effects like headache or nausea).

Well, you can if there is a mental connection to the immune system. E.g. attitudes toward life and toward disease seems to affect outcomes in cancer patients. And placebos can affect that.

That's true, but I'm not sure such a link has ever been established. Is there any study that has found measurable variance in cancer or infection outcomes based on differences in mental outlook?

Here’s an article that speaks to this: https://www.onclive.com/view/markman-column-optimism-plays-a...

In short, yes! Mental outlook has a big effect on outcomes. It won’t cure you, but it will improve your physical outcomes.

This is the meta analysis they refer to: Rozanski A, Bavishi C, Kubzansky LD, Cohen R. Association of optimism with cardiovascular events and all-cause mortality: a systematic review and meta-analysis. JAMA Netw Open. 2019;2(9):e1912200. doi:10.1001/jamanetworkopen.2019.12200

Fascinating, thank you for sharing this, I'll take a deeper look.

It is but it’s a very finicky treatment.

Faith healing has heavy religious tones. Yet not everyone who responds positively to a placebo is religious. So, it feels sort of weird to call the placebo effect some form of faith healing.

I guess it depends on how you define religion. I think that we can have faith in anything - science, your doctor, etc. that could work in a similar fashion to religious faith.

This is specious. Religion is clearly centered around the spiritual and physical beliefs about the world and the practices of a specific group.

Trusting your doctor is usually more along the lines of an educated guess due to the necessity to act without perfect information.

That was the point of the question. Wouldn't you say "placebo" has heavy intellectualizing overtones?

No, I wouldn't.

Why not when you feel the opposite about "faith healing"?

I think part of it is regression to the mean. Like, your body heals itself naturally from many things. Say you get a cold. For most people at some point it'll heal. If you have a group of people with the cold, and try to determine how effective vitamin C or zinc or COLDKILLER777 is, you can't just give it to them and say "look, they're healed", because they would heal naturally. You have to prove that they feel less symptoms or heal faster than people in the same circumstances that don't receive the same molecule.

I also remember some similar stuff for back pain and surgeries. In that context people were seeking treatment when their back issues peaked, and the question was, when you take the cohort of people that had back surgery and the cohort of people that didn't, did the back surgery make a difference? Because some people healed naturally.

I don't know if this is true in that specific context, but to take a more pedestrian one, I've had lots of small cuts, burns and things like that during my life, and they all healed.

The placebo effect controls for all of these factors, and is entirely real. It's the reason placebo control groups exist in the first place. If the placebo effect was just statistical noise, then you could greatly simplify trials by just taking a random group of people as the control, and do away entirely with trying to hide from both patient and technician who is taking the drug - because it wouldn't matter at all.

You can see the opposite effect with the less well known nocebos [1]. People can experience objectively measurable side effects (such as bloating) that are in no way associated with a treatment, but that a patient believes to be a side effect. It can even be fatal. The article references aboriginals who will 'curse' one another resulting in the victim rapidly dying, because he believes so strongly that he is going to die! A similar thing in contemporary medicine has been observed with those who receive a fatal prognosis of cancer with them ending up dying long before there is any way the cancer could have killed them.

[1] - https://en.wikipedia.org/wiki/Nocebo

Isn't that amazing? That we can help people extraneously without having to administer chemicals which have a nonlocalised effect on our bodies.

There's still so much to learn I recently heard that new fathers see a reduction in testosterone. How does having a baby chemically alter a man!? What's the stimulus and mechanism for that...

Pheromones from the pregnant mother seem like a good candidate for the reason behind Low T…

My vote as well. Proximity and "stickiness" to the mother. Likely not a thing if the father moves on to the next region/town after sireing a child. I think back to the isolated humans encountered by the British when working on their rocket programs. Basically groups of women that men moved between, impregnating. Move into a group, kill a child or two and/or fight other males, impregnate someone and move on. I overlay this behavior whenever I try to figure people out.

As a recent father: low sleep.

I can't really refute this but I suspect based on the body of research on this study, the drop must be a lot more than just sleep deprivation. There are plenty of men who work long hours and have poor sleep but I don't believe the drop in T is as remarkable as that post partum

How about the science behind how a baby's crying stimulates milk production.

Again cognitive.

These are only challenging you because you’re assuming cognition cannot affect hormones/chemical systems but we know otherwise.

Not really challenging me, I'm more marvelling at the fact that as giant bags of proteins, we're able to look at our baby and our testicles decide that it's the time to stop doing what they do most of the time.

Cognition is such a handwave IMO - what's the biochemistry behind that? What's the signalling mechanism by which our brain does that? Does that mean with the right external brain signals we can turn off T production?

The implication of cognition having control over the body, which you assert is so well known, is that if we can achieve more control over our cognition we can achieve biochemical control of our body. So the bene gesserit is less fiction than we like to think?

“Mind over matter” is not a colloquialism for no reason. It doesn’t work all the time, but to say it never does is a mistake.

I think that (as with many of the problems of the modern world ;P) it can all be blamed on Descartes.

The notion of dualism is profoundly problematic in a bunch of ways, but the biggest problem with it is that it created generations of scientists who ended up believing that consciousness and body experience are completely separate, which is a little ludicrous when you think about it.

Is it really much different to, say, looking at food and getting hungry, or looking at someone naked and getting aroused? While these are more immediate and conscious experiences, its clear that something as simple as looking at things can affect our biochemistry.

I take your point - you're right maybe there isn't anything that much more different.

But it feels a lot more different because the two examples you used are very clear and direct stimulus-response pairings. See food, get hungry (and get ready to digest)

See mate, get aroused, genitals get ready.

But a hormonal response is weird because:

- you see a baby then your body makes you more nurturing

- testosterone has all kinds of health implications on a longer time frame, it affects decision making, strength, muscle growth etc etc

So the fact that the body has been preprogrammed to alter our mind when we see (our?) baby is kind of impressive. Would be interesting to know if it's all babies or just our own. Do male midwives or fathers of adopted children see the same T drop.

Not to mention that T is also such a sought after hormone that understanding (and controlling) this response could be quite helpful

> Cognition is such a handwave IMO - what's the biochemistry behind that?

Hormones are chemical messengers. They exist to relay messages between different organs and tissues. An organism is constantly sampling its environment and adjusting its internal state to optimize survival and reproduction. Within this framework, it is not at all surprising that cognition interacts with hormones; in fact, it would be kind of surprising to find a biochemical pathway that is entirely independent of cognition.

A good example is stress. For those of us in the first world, stress begins as something cognitive, but it is expressed hormonally as increased serum cortisol. All you have to do to change your hormone balance is start ruminating!

Another example is oxytocin. If another human touches you affectionately, you'll see a bump in serum oxytocin. But it depends on your judgment of how affectionate the contact is, which is cognitive.

I appreciate your response here but it's still pretty handwavy.

Ultimately it's pretty easy to sit back and say - well it's just so. Sure we have a flight or fight and pathways towards adrenaline and cortisol. We also have pathways for oxytocin so we like to find a friend/mate. So obviously we'd also have pathways towards wanting to nurture our kids. That's evolution bro - I guess nature and science are pretty obvious and boring after all /s

The response to children is impressive to me because it appears to be highly specific in its stimulus and the ramifications are pretty large.

Our cortisol and adrenaline response is very generic. All kinds of things can trigger it. You can experience adrenaline by jumping out of a plane or gambling at a roulette table. Even in a showdown in poker or playing video games. So the same response has been conditioned towards all kinds of situations when originally it might have been predominantly towards enemies/predators

Likewise for oxytocin we can probably produce oxytocin from cuddling a pet so it's kind of generic in stimulus.

Meanwhile there's a whole slew of specific responses that men and women have towards babies that I personally find fascinating in their specificity. Maybe because I'm just ignorant but I'm excited for the future when we may be able to understand how a bunch of senses (visual, aural, olfactory) convert into specific neural signals which the body is preprogrammed (how?) to then produce a specific response (via what biochemical pathways) to induce a hormonal response. Which incidentally have pretty wide ranging physiological consequences in the case of testosterone.

I’m unsure why the downvotes, but it would be an interesting study whether there is an adaptation for lower testosterone to encourage fathers to be more nurturing and less aggressive to their offspring. Ie, is there a plausible and measurable explanation for why the t-drop is evolutionarily advantageous.

Who knows. I did a postgrad in medicinal chemistry and I was always amazed at how we scaled up from tiny molecules/proteins into this crazy emergent system. Yet at the individual molecule level there was still so much we couldn't understand from a holistic first principles point of view.

At the time IIRC computer simulations would struggle to model like more 12 water molecules solvating something accurately.

So building up from that atomic level up to proteins into a full physiological system always seemed like magic to me and it's always felt like we didn't really know anything.

Coming back to the t response it would be interesting to know if anything else accidentally triggers it. Eg is it just babies that triggers the response? Only our baby? What about adopted babies or babies of family members? Can other cute things trigger it?

This plus added stress, likely poor nutrition, missed exercise, a whole lot of things happening to new fathers that can lower measured serum testosterone. Whether or not it matters is an entirely different matter. Headline bloodwork numbers don't mean much out of context, and most of the outcomes you'd actually care about (athletic performance, muscle retention, general feeling of wellbeing and energy level) are all impacted by the same things whether or not testosterone is lowered. The one thing that might matter separately is sperm production, but if you care about being maximally able to get your wife pregnant again immediately after she gives birth, you can get that tested separately.

Why couldn’t it just be cognitive? Your endocrine system is not totally isolated from your cognition.

It’s fairly apparent how fathers amped up on testosterone could be worse for offspring survival than those who have a drop, so the evolutionary pressure is pretty clear, then the mechanism is readily explained by “they know that they have a child.”

How does adrenaline get released when you see a dangerous situation with merely your eyeballs?

This seems very much like a mechanism that would be favored by evolution, not an accident of circumstance.

I have no clue how that mechanism migh operate, of course. I realize that makes this comment less persuasive, but so be it.

The evolutionary advantage of it is fairly apparent - but understanding how we can control our own body chemistry without pharmaceutical intervention is of great interest to me.

It's a silly example but what if you could combat age onset testosterone decline with brain exercises or by watching an hour of UFC everyday. I'd take that in a heartbeat over hormone therapy if it worked

The placebo effect is as powerful as many prescription drugs. It amazes me that scientists generally dismiss it rather than actually study it intensely. It's well known that stress can negatively affect your physiology, so I see no reason not to think that the opposite could be true.

Hence why magical thinking can be useful sometimes (or at least that's what I tell myself wishfully)

Can you say more? Does “magical thinking” mean religion, or something else? I’m actually curious.

Can mean believing things will be OK, or events will somehow spare you, whatever the reason. Whether it's because of religious conviction, you've grown a lucky beard, had a happy dream the night before, etc. Even while being somewhat conscious of the irrationality of such beliefs they sometimes can be of help I feel. Also when music inspires / motivates you I feel like it stirs up a similar effects to a strong magical belief.

'Everything happens for a reason' is a pretty common example of magical thinking that you might be more familiar with. Basically when someone favors a magical principal over cause and effect when describing the world.

I don't think it would make sense to say religion in general is magical thinking, a lot of religion can be moral or legal precepts or an explanation of the world that is rooted largely in cause and effect. There is clearly some magical thinking at play when you get into specifics but personally I'm not sure where we would say it enters play: is the belief in a final tallying magical thinking when it is justified by the belief that there exists an entity capable and willing? Not sure.

What do you mean dismiss? It's a pretty well studied area incorporated into most stuff research. What would you like to see additionally?

Please explain the mechanisms behind it then.

There's a massive spectrum between "dismissed" and "mechanisms completely understood". Research continues https://gpsych.bmj.com/content/32/5/e100089

Hilariously, this works even if the patient is aware of the placebo effect. I'm on antidepressants (first time in my life) since two weeks ago. I feel MUCH better already. But the drug I'm taking starts working at least 4 weeks after therapy starts. There was not a single trial where it performed better than placebo during a shorter timeframe.

And yet I feel better.

One thing that doesn't often get appreciated in such discussions is that there are a lot of drugs that seem promising at first, but fizzle out in larger trials. If drug companies had a known pathway to go from positive initial results to very expedited approval, even for limited cases, you can be absolutely sure that they would game the hell out of this system to sell "miracle drugs" to desperate dying patients who will pay anything for a chance.

While it's sad and horrible to know that a cure for your condition may already exist and be just out of reach, and I can imagine the despair at that, I'm not convinced the alternative is all that more appealing.

I would also note that it's certainly not, by any stretch, the worse injustice in the medical system. For every one patient with a terrible cancer that might have survived if allowed access to an experimental treatment, there are millions of people dying of easily treatable diseases for which we have had a treatment for the last hundred years, but who can't afford it.

The existence of a cure for your condition that you just can't access for whatever reason is a reality of our system. Caution in introducing new drugs is actually one of the more rational reasons, that one needs to try to come to terms with.

Got to remember that there is a great financial incentive for trials to come out positive. Who pays for clinical trials?

> One doesn't need elaborate phase 3 RCTs to figure out if there's a good shot that a treatment is working....The existing system raises costs and causes people to die while waiting a decade or more for exciting treatments

The FDA often approves cancer drugs without a phase 3 randomized trial. In fact, most new cancer drugs are approved without a phase 3 trial.

Just taking a random cancer drug from this list: https://www.fda.gov/drugs/novel-drug-approvals-fda/novel-dru...

"The efficacy of IMDELLTRA was evaluated in Study DeLLphi-301 [NCT05060016], an open-label, multicenter, multi-cohort clinical trial....A total of 99 patients received IMDELLTRA..."

This is a new small cell lung cancer drug approved via a phase 2 study that didn't have a control arm and wasn't blinded. This is pretty typical.

> one can see it in tumor response and comparison to known KMCs.

Anything measured by a human can be biased by knowledge that a patient received a treatment, including tumor response (often blobs on a screen from a FDG PET/CT scan.)

RCTs are the gold standard. We don't need to start chipping away at the rigorous standards we have in place to accurately measure the value that a medicine offers.

What we can do - and are doing right now - is do a risk-benefit analysis and allow drugs to be approved with a weaker set of data so that patients with a life-threatening illness can get access earlier.

There is way too little obsession with them given how much of research isn't using/reusing this method

And specifically regarding cancer we also know a lot of very extensive drugs fail at reducing mortality

(specifically, as far as I remember, tumor reduction may have no connection to mortality for some cancers, so we don't really "know what happens" without factual data)

Furthermore, RCTs just identify the substances that have positive effect on the largest subgroup of the studied group (which 'coincidently' is what the Pharma Mafia is most interested in). But we are all individuals, differing in genetic, epigenetic, foreign organisms, metabolism. The near only value the RCTs have to an individuum is to determine the substance(s) which most probably have an effect.

Memorably, someone in the UK recently dismissed over 100 studies because they didn’t perform double blind RCT. Which is hilarious when you consider the studies were about gender affirming/reassigning treatments.

100% agreed. "RCT's are the gold standard" doesn't make them gospel.

Reply on news.ycombinator.com