Embracing Bayesian methods in clinical trials
jamanetwork.com
jamanetwork.com
the reason it has been limited to those cases is
drug development, today, is constrained by commercialization.
all four categories listed - ebola, sars-cov-2, pediatric and rare disease drugs - each for their own reason, have low commercialization risk, so if they are scientifically robust solutions, there is ROI.
most drugs in development today are new indications for existing molecules, generally in oncology, also because these are the most favorable conditions for commercialization.
commercialization, commercialization, commercialization. people don't want innovations in drug approval, nobody is clamoring for that. they want innovations in commercialization.
i don't know if bayesian methods will make trials a lot cheaper. and that's their problem! there are already a lot of very smart people working on this issue, and they have litigated to death all the objective facts.
When they are working with a pharmacy and your insurance, the price they'll charge you is $10,000 for enough pills to last a month. But when your insurance says "We won't cover that" all the sudden you find out the company has a backdoor subsidization program which will fully cover the cost of the drug for reasons I can't really fathom (good will?)
What's even more uncomfortable is my insurance (aetna) also mandates that I get my medicine from their subsidiary, CVS.
I really don't like this sort of thing. The price of everything in medicine feels distorted in unbelievable ways. Like famously a $0.25 acetaminophen pill that somehow magically costs $10 in the hospital. I guess there's some nice individual packaging.
I'm less and less convinced that it's even a cost sort of thing with these pharmaceutical companies. Like, sure they'll love to reduce that as much as possible. But the price itself seems entirely fictitious and based on what they can commonly get insurers to pay, and not anything related to the actual R&D of the drug.
Probably you are a soft recruit to "they got better' numbers for post trial marketing but even then, it's like a Ford is $30,000 or $3 depending how easily it can be sold.
okay well, let's say you were to model "what explains the price of something" and you had two factors in your model, "markets" and "politics." most good models will have a residual which means "everything else."
when i say "low risk" to commercialization, i could mean a lot of things. it depends on what the drug category is.
if you develop the only cure for an otherwise fatal, pediatric, congenital disease, the parents will be willing to pay an unlimited amount of money for it. so there's no market price for it. so we know that, no matter what, a correct model for that is, 0% market. this is really, extremely difficult for most people who look at this issue, because it renders their opinions about what should you pay for this drug basically moot. they'll all have different answers. so the numbers of our independent variable, "price people are willing to pay" will be uncorrelated with the price people actually pay. and this is one of those instances where the absence of correlation means no causation. instead of looking cogently at this situation and saying, "well, okay, maybe i should temper my outrage about drug pricing in this actually very important scenario" they just get more outraged.
so what factors are left: political and everything else? well, the drug is effective, it's a cure, this describes a lot of drugs. that's what we are talking about, effective drugs. the FDA process has ALSO attacked the importance of market pricing from this angle too, the clinical trials process here and how it has been standardized around the world basically works. so the residual, if it's about the efficacy of the drug, actually does NOT explain the price very much at all. this is counterintuitive!
this is why a CURE for a disease is not 10x more expensive than something that already exists and manages a disease. so...
what's left? my guess would be, the price would be explained by, okay, a lot of it will be political. like 90% political. this is what i mean by "low risk of commercialization." nobody is going to win an election being like, your kid should die.
are you getting it now? you are getting hung up on prices. you don't know the first thing about prices. you think the journey to understanding prices has to do with "$0.25 acetaminophen pill that somehow magically costs $10 in the hospital" and hospital charge lists or whatever, which is 200% wrong, that's a huge red herring.
prices in categories that are low commercialization risk are almost always explained by a political process. which is to say, all the innovations today in commercialization are political. that's bad. but that should also illuminate why, if I were running the FDA, that's where my focus would be - besides, at the rest of the HHS, 10x the basic science R&D budget.
Two of them were paying significantly more for their support than the others. That's because those two had their phone numbers set to ring even when my phone is in quiet mode. Just for the privilege of being able to wake me whenever you want, and get my attention even when I'm in the middle of a hike, you're paying substantially more.
When my first child was born the nurses had a terrible time drawing blood, so they gave him sugar water to calm him down billed at $40/5ml.
It’s because of the way insurance works in the US. Insurance companies have formularies, which are essentially menus of what products they cover. They also will label certain medications as “preferred” and actively steer consumers to them. Pharma companies fight to get preferred coverage from insurers.
Because of this, pharmaceutical companies go through complex negotiations with insurance companies. In theory, this is to get pharma companies to compete on price and offer discounts (called “rebates”). An industry of middlemen called pharmacy benefit managers (PBMs) has arisen who negotiate with pharma companies on behalf of the insurers and create the formularies. The problem is, these guys take a percentage cut of the discount they secure for the insurers. This creates a perverse incentive to give preferred status to more expensive drugs. Take this hypothetical example of two competing drugs:
- Drug A costs $150, but gets negotiated down to $100 - Drug B costs $200, but gets negotiated down to $100 - The PBM gets a fee of 10% of the secured discount, meaning they make twice as much for creating a formulary with B rather than A
To no one’s surprise, drug B gets given preferred status over A.
Pharma companies figured this out a long time ago, and began to jack up their prices each year, only to immediately negotiate them back down to where they were previously, in the form of rebates, because this increased the likelihood of getting out ahead on the formulary by the PBM. The best example of this might be insulin, which has skyrocketed in list price, but profit for the insulin companies has actually remained much more stable. This is because each year the sticker price is raised, and then immediately slashed in the “negotiations” with the insurers.
After a while, the pure sticker shock began causing a lot of justified outrage among patients and the public. Pharma companies saw they were taking the blame for skyrocketing prices even though at the end of the day they were negotiating these prices down and weren’t actually making that money due to rebates. So they began to offer alternatives for customers not on insurance, in the form of “coupons”, “savings cards”, etc. These often get you prices close to what the actual cost of the medicine is before the “jack up the price then rebate it down” dog and pony show. But insurers/PBMs reacted poorly to these, and began punishing pharma companies through the formularies. This is why these have become shadowy “backdoor” programs.
It’s also important to note that the PBM’s aren’t even independent middlemen. 80% of the PBM industry is dominated by 3 companies, all of whom are owned by insurers: Express Scripts (Cigna), CVS Caremark (CVS/Aetna), and OptumRx (UnitedHealthcare)
So why even go through all this?
- Pharma companies don’t really have a choice, they have to to get on the formulary
- PBMs make their money entirely through this scheme
- Costs for insurers aren’t ultimately changing much from year to year. However, higher sticker prices means the public is ever more dependent on insurance for medical bills. It also provide justification for increasing premiums more than their costs might otherwise. And pharma ultimately takes the brunt of the blame.
Lastly, if you are wondering why this doesn’t exist in other countries, it’s an unsurprising reason: government subsidized health insurance. Unlike insurers and PBMs, there are no perverse incentives or profit motive, so cheaper prices are an actual benefit. Medicare/medicaid doesn’t have to go through these shenanigans, but they aren’t available widely in the US. You don’t even need healthcare for all. You just need a public healthcare option for everyone. That introduces an actor whose motives actually align with consumers, invalidates this entire charade, and forces insurers and PBMs to actually compete on merit and price.
TLDR: The lack of public healthcare options in the US has created an insurance cartel that has both consumers and pharmaceutical companies by the balls.
That's a good observation, but I think it's an incomplete picture. Another important constraint is often regulatory inertia and historical baggage.
The UK pioneered small classical and adaptive trials using Bayesian methods, and there were some promising results. A lot of modern Bayesian methodology was, in fact, developed at the MRC BSU Cambridge with this goal in mind. For example, the probabilistic programming language BUGS (1989).
Given that most drugs fail, the industry is highly incentivized to use Bayesian methods to fail faster. These models allow for more rapid dose-finding and the ability to distinguish promising leads using interim data, which is vital given the massive cost of any trial, especially late-stage failures.
But for Bayesian methods to make a dent, they'd need to be applied to a large number of trials, and change doesn't happen overnight. Lots of big pharma players, e.g. GSK, are becoming interested in moving to Bayesian methods in order to leverage prior information and work better within small-data regimes.
The problem with this approach is that we can only observe ONE level of treatment effectiveness, i.e., the level of treatment effectiveness that the treatment actually possesses. All other possible levels of effectiveness are entirely hypothetical. There's no data about all these other possible levels of effectiveness because they don't occur in reality. So the data cannot possibly tell you anything about how likely is the observed outcome, because the observed outcome is the only outcome that you observe. I
This criticism was made over 100 years ago, and Bayesians still don't have an answer. They just keep going as if nothing happened, but the reality is their methodology is utterly and fatally flawed.
This could also be viewed as supporting the Bayesian perspective, where the observed data are not viewed as random variables - they are fixed. This is because, as you say, the observed outcome is the only outcome that you observe. It is the classical setting, in comparison, where we instead do our analysis by treating the sample as a random variable, placing the counterfactual on other non-observed values ("what if I had drawn a different sample?"), even though we didn't. Bayesian methods treat the data as gospel truth, and place the counterfactual on the different parameters ("what if the population were different?"), even though it isn't.
The other criticism you have is
> The problem with this approach is that we can only observe ONE level of treatment effectiveness, i.e., the level of treatment effectiveness that the treatment actually possesses. All other possible levels of effectiveness are entirely hypothetical.
This is true of both Bayesian and classical methods. We build models that would explain how different hypothetical levels of effectiveness would affect what data we should expect to see - that is the whole point. Classical methods also involve exploring scenarios in which purely hypothetical values of the parameter may be potentially true, and characterizing counterfactual samples that could have been drawn from them, even though in real life they couldn't have been.
I wrote another comment here clarifying my point, if you're interested: https://news.ycombinator.com/item?id=47566033
In Bayesian inference, the population ratio is seen as a quantity that can take different values each with a associated probability (i.e. a random variable), and the result of Bayesian inference is an estimate of the probability distribution of the population parameter, in this case the population ratio. Now, in reality the population ratio is a concrete number, say 9-to-10, meaning that there are 9 men for every 10 women in the population. But Bayesians don't care. They'll tell you that the population ratio is a random variable which can take many values, and that the probability that it is equal to 9-to-10 is whatever number between 0 and 100%.
This is nonsense because the population ratio is NOT a random variable. People don't come in and out of existence randomly, right? In a way, they're saying there are infinitely many possible universes, each with a different population ratio, and then they come up with an estimate of the probability that the universe in which the ratio is 9-to-10 has whatever probability of occurring. This is absolutely BIZARRE. (I hope you agree). And it's wrong because it's impossible to know how likely one universe is compared to all other possible universes, since we live in our universe and this is all we can hope to observe.