A Dying Scientist and His Rogue Vaccine Trial
wired.com
wired.com
Could there be a series of exams that a would be patient would have to write, to demonstrate that they truly understand the risks, and what protocol is to be followed?
In investment they have an "accredited investor" category to avoid higher risk investments from being offered to those who can't afford the risk.
If you have a chronic disease and pass the exam then you become an "accredited trial patient".
2. You weren't dying before, but you are now.
3. Not dying, but the extreme agony is back.
4. Still not dying and no pain, but you now look like one of the aliens from Close Encounters. Career choices are limited.
...
Well that does sound like a good reason for Congress not to sign on the dotted line, given the definition of 'it'.
- there are plenty of quacks who already con people - people who do actually believe their product works, but it doesn’t - things that may work but also may kill you (remember that drug trial in London a few years ago that had got through all pre-human trials without a problem?) - placebo is real, and the pre-approval access ideas generally are a pay-to-play concept, and I’m not sure people would want to pay for a blinded treatment - people talking about these trials always seem to say “once basic safety tests are done”, but phase 1 and 2 are the basic safety tests. Phase 3 require enough people that drug companies are often actively working to find participants.
The real problem here is that universities etc keep posting hyperbolic articles saying “we have found something that may cure X”, with the rider for studies and fda approval are necessary.
Laypeople interpret that as meaning the fda approval is preventing them from accepting patients for approval, when typically the component in question is not remotely close to a treatment:
- it’s just binding to a target invitro. -There’s no information on what happens one it has bound. - there’s no evidence it can even get to the target invivo anyway
- animal trials are only advisory. There have been many super effective drugs and vaccines that have no effect in humans
- can the get past the immune system? Will the immune response kill a person
- I’ve seen articles about compounds that would (and turn out to be) toxic
Etc etc
But all a patient sees is the “may cure X” bit and so demands a right to use the drug.
That’s why these things are a bad idea
I asusme you mean the TGN1412 trial [1,2].
For the most part, all this is just a waste of time and money. But for medicine, things that don't work create an opportunity cost--someone uses an anti-oxidant-packed cancer treatment and dies. It's a basic information asymmetry that exists in the market: its too hard to prove that things work, and the wonders of modern marketing ensures that lots of things that don't work still sell like crazy.
Efficacy tests address that problem. But proving efficacy is hard and expensive. But they're probably not "too expensive." That is to say, if you made clinical trials cheaper by just testing for safety, you'd probably suffer a net economic loss due to people spending money and dying from products that don't work than you would on the trials.
So why are they approved? Because there's nothing better. It's the same here, make these trials easier for conditions that otherwise have no good options.
The FDA system is the only way to get therapies to large numbers of patients. But it isn't the only way to get therapies to patients period, and it has serious flaws. The FDA is risk averse to the point of suppressing progress. The costs have doubled in the last ten years for no good reason.
There is a desperate need for a less heavy handed, more distributed, faster system that favors pace of progress - but all attempts to build it will run into this problem of incentives. The organizations running trials are targets, and people will respond poorly to being struck by risk no matter what they sign or are given to understand up front.
Non-profits may work for low-cost treatments, such as repurposed cheap-to-manufacture compounds from the drug databases (e.g. senolytics), but some types of therapy genuinely do require an organization with tens of millions in the bank to sustain development. How do they proceed if not via the FDA?
One of the younger girls who was with her in the treatment was completely disabled, bound to a wheelchair, couldn't talk, could barely see, and on day four after the treatment she got up, could walk and talk. A true freaking miracle of science. That girl now does mountain biking and mountain climbing, fully recovered.
What kind of treatment? What country? Where can I find more about this magical therapy?
There are small trials in multiple countries, but very hard to get in.
After that, i think it's fine to try an untested drug on consenting patient with a severe chronic, or an acute life-threatening condition.
I understand why it isn't done. Market forces forcing companies to save money on trials or die, may quickly unwrap this situation to Dr. Mengele-style 'trials' if not kept in check.
The vaccine from this story seemed to be safe and effective for some participants, but nearly killed another. How can we differentiate the people that it will help from the people that it could kill? This is a real problem in FDA approved medications. Just look at the side effects list for some of the "biologic" TNF blocking treatments for autoimmune diseases. Yet they are beneficial for most people.
There's a reason this research takes so long. Things might have gotten to market sooner decades ago, but it all comes down to a trade-off with potential good vs safety. When political maneuvering gets in the way, you can get things like the Vioxx disaster. Even with Vioxx, FDA monitoring and procedures did do what they were suppose to in identifying the problem and pulling it off the market.
The biggest problem really seems to be money, resources and funding the research. You might discover something promising, but your company will need to pay all those people, buy equipment and purchase animals to go forward. If you get to monkeys and lose a single one, you gotta go back to the beginning. You might find a side effect in 5% that's really bad/serious and your entire compound might need to be scrapped.
Companies need to make money off of those investments so the compounds that do succeeded might become insanely expensive. The entire process is pretty backwards and I know people who are biochemists in this field who get paid really shitty despite their important work. A lot of money goes to equipment, but a lot also goes to management and cronies in the shops that can fund this stuff.
It's a mess. There needs to be a better, not-for-profit model where a failed compound doesn't matter as much.
Agreed. We’re way past the sane point on the trade off. Aspirin would not pass FDA review under the current regime. Efficacy testing doesn’t appear to do much good but it clearly makes getting a drug to market slower and more costly.
For more on this see http://www.fdareview.org/
Come again, it's not important that any medication performs as advertised?! The purpose of Phase II testing is to determine if the new compound is better than placebo; the success rate is ~ 30 %, 70 % are no better than placebo. 70 % of candidates are as good as taking nothing at all.
In a public health system this is the most important question to be answered: does the new treatment offer a greater benefit to patients - if not you can't ask the public purse to pay for it.
I can also see the risk of backlash - the public perception of bigpharma is already low; if it begins to offer expensive treatments of low value trust in the institutions will erode further.
We have a number of drugs in the same class that are as or more effective with less frequent adverse effects, and the same price to market. It is also one of the leading causes of toxic ODs presenting to the ED. Aspirin shouldn’t pass as an over the counter Anywhere, and wouldn’t but for historical inertia. Don’t confuse familiarity with safety.
> efficacy testing doesn’t appear to do much good
Efficacy testing is literally how we determine whether a drug works, and what the magnitude of its effects are. Without it, we’d have no idea if a drug worked, if it worked better or worse than competitor drugs, if it worked better or worse for some conditions or sub populations, etc. It’s -the- key information generating step on drugs. This comment is ridiculous to the point of caricature.
>fdareview
A site from the far-right, unknown-anywhere-but-Fox-news Independent Institute. Can you guess before popping the link open whether they advocate on lightening regulatory burdens on pharma companies?
>We have a number of drugs in the same class that are as or more effective with less frequent adverse effects, and the same price to market. It is also one of the leading causes of toxic ODs presenting to the ED. Aspirin shouldn’t pass as an over the counter Anywhere, and wouldn’t but for historical inertia. Don’t confuse familiarity with safety.
Could you add some more data to back up your statements? For example, you state "It is also one of the leading causes of toxic ODs presenting to the ED"
>...During the latter part of the 20th century, the number of poisonings from salicylates declined, mainly because of the increased popularity of other over-the-counter analgesics such as paracetamol (acetaminophen). Fifty-two deaths involving single-ingredient aspirin were reported in the United States in 2000; however, in all but three of these cases, the reason for the ingestion of lethal doses was intentional—predominantly suicidal.
https://en.wikipedia.org/wiki/Salicylate_poisoning
Considering that literally thousands of tons of aspirin are consumed every year, this safety record likely compares well with other drugs.
As a comparison, the most commonly used alternative to aspirin is acetaminophen and that is certainly much more dangerous:
>...Many over-the-counter and prescription-only medications contain paracetamol. Because of its wide availability paired with comparably high toxicity, (compared to ibuprofen and aspirin) there is a much higher potential for overdose.[88] Paracetamol toxicity is one of the most common causes of poisoning worldwide.[22] In the United States, the United Kingdom, Australia, and New Zealand, paracetamol is the most common cause of drug overdoses.[17][89][90] Additionally, in both the United States and the United Kingdom it is the most common cause of acute liver failure.[91][7]
https://en.wikipedia.org/wiki/Paracetamol_poisoning
>...A site from the far-right, unknown-anywhere-but-Fox-news
I agree with you that efficacy testing of potential new drugs is important, but adding a statement with pejorative labelling like this feels somewhat like you are simply trying to poison the well.
The most recent public data from the american association of poison control centers is 2016. For toxicity events alone they don't break out analgesics into sub-groups, but for fatalities acetaminophen is in fifth place with 5.39% (there's also acetominophen combinations but those really don't count - they're all opioid combinations, and really represent an entirely different phenomenon). Aspirin is in 16th place, with 1.91% of fatalities. For scale, opioids are 8.3%, and anticonvulsants are 1.76%. Most of these are multi-substance exposures. For instances where there was only a single substance exposure, acetaminophen was about 12% and aspirin 3% (opioids 6.4%, anticonvulsants 0.51%). Ibuprofen, with 81k case mentions (vs. acetominophen's 58k and aspirin's 17k), didn't make the top-25 killer list.
I didn't dig up the AAPC article mentioned in wikipedia, as it's about two decades old now, but the AAPC annual report I'm looking at right now shows most (~13k/17.5k) sole aspirin exposures were in the <12 crowd. It's not a big demographic for suicide.
As to my statement that we have better drugs, the Oxford Pain Group has released the Oxford League Table for relative comparisons between NSAIDs, based on a meta-analysis of comparative trials. With a standardized measure of "50% of pain relief over 4-6 hours" as a successful treatment, the league table ranks NSAIDs by the number of patients you need to treat to achieve said outcome for one patient. Aspirin 1200s (.3 of your max recommended daily dose) are in 16th place, with an NNT of 2.4. Acetaminophen 1000s (1/4 of your daily max dose) are in 34th place with 3.8. Ibuprofen 800 (1/4 of your daily max dose), drug of choice for every doc I know, are in second place with an NNT of 1.6.
>Paracetomol wikipedia tox
I didn't say aspirin was the single biggest killer. I said we had other drugs "as or more effective with less frequent adverse effects, and the same price to market," and that "it was one of the leading causes," not the single biggest cause. I don't think the wikipedia page really contradicts that?
> but adding a statement with pejorative labelling like this feels somewhat like you are simply trying to poison the well.
I disagree. If someone links to a peer-reviewed article in a respected journal and the response is "that author is a shill," you're coming down on the side of ad hominem. But when someone links to an NRA website and says "you should read more here about gun safety here," ... it's entirely appropriate to point out the source of that information. Lobbyists, propagandist "think tanks," and other highly partisan groups that build a business model on trotting out talking points for their respective parties absolutely deserve being identified on sight as such.
One obvious issue is that this data isn't the complete picture - obviously the amount of consumption of the drug is the other half of the picture. I've seen estimates that over 100 billion aspirin doses are consumed each year of aspirin - I doubt that much is consumed of other drugs you listed. If aspirin was no longer available over the counter, wouldn't that just increase the risk of overdose for its competitors? (Particularly for acetaminophen).
In terms of safety,. the FDA has also singled out Ibuprofen and other non aspirin NSAID drugs for: health risks that (they claim) aspirin doesn't have:
>...People who take nonsteroidal anti-inflammatory drugs (NSAIDs) (other than aspirin) such as ibuprofen may have a higher risk of having a heart attack or a stroke than people who do not take these medications. These events may happen without warning and may cause death.
https://medlineplus.gov/druginfo/meds/a682159.html
Any of these deaths caused by other NSAIDs would likely not be attributed to Ibuprofen on the death certificate.
In terms of efficacy, you wrote:
>...Ibuprofen 800 (1/4 of your daily max dose),
3200 mg is the daily max dose when under the care of a Doctor. The instructions on the OTC bottle will say to use 1200 mg per day. So as an OTC drug (which is what we are talking about here) that is more like 2/3 of the suggested daily dose.
>…I didn't say aspirin was the single biggest killer. I said we had other drugs "as or more effective with less frequent adverse effects, and the same price to market," and that "it was one of the leading causes," not the single biggest cause. I don't think the wikipedia page really contradicts that?
What you originally wrote was:
>…Aspirin shouldn’t pass as an over the counter Anywhere, and wouldn’t but for historical inertia.
Aspirin is likely available over the counter in just about every country in the world. Considering the massive usage of this drug, the adverse incidents seem low. The number of deaths from overdose are minuscule compared to the usage and I think you agree it is safer than other OTC counter drugs such as acetaminophen. In terms of efficacy as an OTC drug it seems comparable to the competitors.
>...But when someone links to an NRA website and says "you should read more here about gun safety here," ... it's entirely appropriate to point out the source of that information.
You weren't just pointing "out the source of the information" - the group behind the info is on first page, that wasn't a secret.. What you wrote was:
>…A site from the far-right, unknown-anywhere-but-Fox-news Independent Institute.
Terms like "far-left" and "far-right" are often casually used as a pejorative against a person or group. Neither of these terms is well defined, but most people consider "far-right" to be related to Nazis, skinheads, etc. - i.e. (possibly dangerous) kooks who should be ignored. Looking at the wikipedia entry for the Independent Institute:
https://en.wikipedia.org/wiki/Independent_Institute
I don't see any evidence that they are a hate group. You also wrote
>…unknown-anywhere-but-Fox-news
Are they often quoted as a source on Fox news or a guest on a Fox show? From the wikipedia page it sounds like they would be in disagreement with Fox news on various issues like the Iraq War.
Intentional or not, trying to shut down discussion by labelling someone you disagree with like that is poisoning the well. Far too many members of the public just stay in their ideological bubble and ignore any arguments except those they agree with. It is easy for the average person to be skeptical of ideas they disagree with, it is much harder for them to be skeptical of ideas they want to believe. There is nothing wrong with pointing out disagreements with an argument someone makes (as you did earlier) or pointing out possible conflicts of interest that weren't disclosed by the authors, but labelling someone you disagree with as a kook that should be ignored on principle isn't really the same thing.
Phase II/III clinical testing is expensive, and the failure rate (i.e. low efficacy, compound simply doesn't perform as expected, and certainly no better than what's already at market) is high. That's what it is, there isn't much that can be done at that front.
Thankfully a few months ago Goldman Sachs directly addressed the elephant in the room: pharma yields bad returns, investment in pharma is inadvisable. There needs to be a different financing model, perhaps from the public purse.
How does that solve the issue at hand? As you said, development is expensive and has a high failure rate. Why would public funding change either one of those?
Right now science can't do anything about the high failure rates in Phase II and Phase III trials. The problem isn't to come up with a compound against your target of choice, it's that apart from clinical trials we don't have a way to predict if in real life it will perform better than placebo (Phase II) or adequately (Phase III).
The development of Fidaxomicin/Difficid is actually an example of a fairly inexpensive clinical trial (by clinical trial standards), and Merck didn't buy Cubist for $9.5 billion for a drug they don't think will recover their outlay.
science staff being underpaid relative to the value they provide is the norm. everyone knows the scientists are there for the love of it.
the equipment is pricey, sure. million dollar machines, yada yada yada.
but the marketing budget is far higher than the r&d budget. it isn't even close.
and yeah, it's a mess. from my experience on the inside, companies are willfully deceptive at all phases of the research and clinical trial process. failures are swept under the rug as much as humanly possible. especially if those failures endanger a real person.
if a person gets hurt in a trial, i guarantee you that there was knowledge of the risk that was drastically downplayed.
drug companies test tens of thousands of candidate drugs before moving to trials. in the last round of testing before trying to move to clinical trials, the drugs are typically very similar variants of each other. problems are found in most. the one candidate where problems aren't found, likely by luck, is the one that advances. survivorship bias is intentionally played up as evidence of validity. then, of course, the "black swan" negative event happens. stock price plummets.
cue statements regarding how much they care about trial patients, and the problem couldnt have been foreseen, etc etc.
but they knew all along that it would happen. it was happening to mice with the drug's sibling candidates which didn't make it to the clinic.
If you go into cardiac arrest, you may or may not be given epinephrine. Some hospitals use it in their resus protocols, some don't. Nobody knows whether it works, but it's the subject of heated debate at anesthesiology conferences. Some trials suggest that it's marginally effective, others suggest that it reduces long-term survival rates. There's conflicting evidence on timing and dose. As you teeter on the threshold between life or death, someone might be giving you a drug based on decades-old studies performed on dogs, some clinical guesswork and a bunch of inconclusive research.
For the price of one drug that offers a few extra months of life to a handful of patients with a rare form of cancer, we could resolve several of these clinical debates.
http://anesthesiology.pubs.asahq.org/article.aspx?articleid=...
The medical community has realized this, and been tackling this problem for a few millennia. Hippocrates addressed the topic.
>and figuring out why treatments that are safe and effective for some people are unsafe for others.
That’s the hard part. It’s why we pay attention to how we construct patient recruitment, sample demographics, assess the metabolic pathways by which a drug is processed and study genetic variants in those pathways, develop biologics with different types of chimeric proteins, retrospectively analyze adverse events for patterns, etc.
Each and every one of those things that are standard elements of modern medicine and rational drug design are about understanding variability in drug response. We haven’t perfected this yet - nor are we close. Anyone who wishes to lend a brain cell is welcome.
I was told the opposite the other day:
"the fact that clinical trials seek people maximally healthy but for the target conditions rather than a representative sample of those with the target condition is a very frequent complaint." https://news.ycombinator.com/item?id=16868660
>"Anyone who wishes to lend a brain cell is welcome."
Its pretty misleading to promise this. People who disagree with the "publish an endless series of statistically significant differences" paradigm are not going to find it at all welcoming.
Those aren't quite in contradiction. So, where you start, is with a trial that tries to eliminate meaningful comorbid conditions to allow it to answer the question: "Does this drug work for Disease X?", rather than the question "Does this drug work for Disease X when accompanied by Diseases Y and Z?" There are just far, far too many permutations of X, Y, Z, etc. to allow them all into a study and even pretend to have answered the question about the drug. So, it's not where you start. But, you're absolutely paying attention to patient demographics here, that's part of the point: can we come up with a reasonably homogenous patient pool for step 1? That's not really about being "optimally healthy," it's about homogeneity, except where the latter implies "not a giant variety of comorbid diseases."
But, as I hope I've made clear in the way I reiterated it, that's the initial step. Because, hey, we really do need to answer the question about Drug A for Disease X before we start working on the various permutations of Disease X with other diseases. The latter is usually studied through retrospective analysis rather than RCT. Why?
A given study has only a certain sample size to work with. There's this sort of odd implicit idea that small sample size is merely an issue of study design, but not really: in practice, a research group only has a catchment area of a certain size - they'll only be able to catch so many patients per unit time, and you can't run a study forever (both because you run out of grant funding and because the state of the disease and therapy moves on.) The more specific you get - e.g., people with Disease X and Y and Z, but neither Y and Z to be more than moderately severe - the fewer patients you can potentially recruit, to the point where the final result is statistically meaningless. This isn't really the worst thing ever: the less capable you are of recruiting patients because of their rarity, the less useful such a study would be to the public because of their rarity. As a practical effect, you can't really build a study to control for every meaningful confounding comorbidity. So, we tend to wait for data to accrue and analyse it retrospectively. It's not as good as an RCT, but it's the best we can do under practical constraints.
It's one of those things worth criticizing, in theory, but continues because we don't have the means to do better. Ideally we'd have studies of 20k+ people selecting for significant sub-groups of each permutation of common comorbid conditions. That's just beyond any hope of organizing, running, or paying for.
>>"Anyone who wishes to lend a brain cell is welcome."
>Its pretty misleading to promise this. People who disagree >with the "publish an endless series of statistically >significant differences" paradigm are not going to find it >at all welcoming.
That's simply untrue. I know people working in operations research who are developing novel predictive models for ED patients to predict where they'll need to be sent after workup (and how to prep the appropriate hospital dept. for their arrival, and to estimate when that will be); the VA does shit-tons of operations research. I know people working in the basic sciences who are working on developing ever-more-sophisticated models of the current state of physiology/biochemistry, to allow for increased accuracy of predicting a drug's effectiveness and toxicities before moving on to in vitro testing. And an interesting negative result is about as easily published these days as an interesting positive result (even Easterbrook's big '91 paper on publication bias found that it was for observational and lab studies, but couldn't identify bias in RCT publication.)
The full variety of interesting work to be done in clinical trial design isn't super obvious from the outside. Especially since the folks moaning and whining about it tend to be academics, and a big chunk of good clinical trial work doesn't involve many university academics, and the other cool work (e.g., operations research) is being done in industry and at the VA. But "not super visible to university academics, who are the ones talking about publish or perish" and "non-existent" are very different things.
>"(even Easterbrook's big '91 paper on publication bias found that it was for observational and lab studies, but couldn't identify bias in RCT publication.)"
I'm not familiar with the Easterbook 1991 paper but that is a pretty strange thing to claim since it is even one of those criticisms that academia has managed to turn into it's own area of study. It didn't look closely at any of these papers but just for example:
>"we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. [...] Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. " https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3702538/
>"We conclude that RCTs investigating antipsychotic drugs suffer from substantial outcome reporting bias" https://www.nature.com/articles/tp2017203
Vaccination is positively ancient technology (Smallpox circa 1796), but it is still difficult to say why e.g., some vaccines are broadly effective at producing immunity and some aren't (and what the factors at play are in the recipients), or why we can produce successful vaccines to some diseases and not others.
A specific example: the efficacy of BCG vaccination (tuberculosis) varies by manufacturer and the reasons for why that is are still unclear.
There are a number of large NIH-funded initiatives that aim at understanding the complex cellular and molecular interactions that result in immunity as a result of vaccination under the Human Immunology Project Consortium (a bunch of high profile papers in Cell, etc. over the past few years).
Opinions are like yada yada.
Because, our field is not as set as medecine.
It's hilariously full of "well obviously the solution to X is Y" for things outside the author's area of expertise.
I’ve begun actively avoiding them because of how unpleasantly off-base they are. No one wants to read an entire thread of, well, what you imagine doctors would say if they were discussing programming.
As I wrote in another reply, I guess I was wrong about assuming humility in others.
The HN community includes quite a few physicians, by the way, who often add excellent insight to threads. Here is an example from yesterday: https://news.ycombinator.com/item?id=16983369.
My son, found when he was 7 and died when he was 12, had cancer and the idea of a trial can seem almost magical unless.... My son had bone cancer. To be in a trial you have to keep the tumor so you can measure effectiveness. The ONLY cure for bone cancer is surgery. So I hate the way trials work for my son and other's like him. This is why there has not been a single change in chemo or life expectancy for over 30 years for bone cancer!!!! It looks like finally dog research (More money goes to finding cures for dogs then children, let that sink into you when you have a kid with cancer), has the possibility of actually changing this trend.
Now the other issue is your child has cancer and they are only trying to see dosage. So they really want to know how much you can tolerate and not die. Again not a fan of putting my dying kid through more side-effects. Many kids choose to forgo treatment and just die, and you as a parent have to decide do I make the decision for them and every throw up and every pain is your fault (I had to make that decision once and I was glad I did he lived a few more years). I know some parents who agreed with their 5 year old and stopped everything but pain relief. I also have friends who pushed through years of treatments and they still suffer from the experience.
So those who did trials were soft tissue patients. One was with neuroblastoma and one has medulloblastoma (Brain cancer of the brain stem). The girl with brain cancer is still with us 3 years after it relapsed and her trial was not an effectiveness but a dosage but it looks like it has added years to her life already.
So I feel like this kind of discussion is good on HN.
Now my mom took my sister who had medulloblastoma to Mexico and I don't think it did anything but separate us from our family. I know some that I really think that crap killed them early and the stupid diets and painful procedures and false hopes just are horrible. I kind of put this who thing into that category. Your at the point where you will do anything, but in the end you would have better off not even do anything.
Personalized filtering might help with that. And thus help address eternal September effects, of bad content driving out good. Even with HN tech posts, I notice I've a 'this discussion is too reddit' threshold - an "ok, this could have been an interesting discussion topic, but there's too much low-quality discussion already here, and tilting at 'someone on the Internet is wrong' isn't a happy thing, so I'll skip this one".
There are a lot of HN-like communities of expertise across many fields. Public and private. They are often a good place to get news commentary. With the exception of a few reporters who are part of such a community, and write for a more general forum (sometimes with community feedback on their reporting), general news media (NYTimes et al) feel to me like reporting after a foreign bureau has closed - lacking a reality check, insight, and nuance.
Getting better at this seems to be one of those societally critical technologies for which the incentives are poor, so there's little progress decade after decade. For new discussion systems, perhaps being better at insulating experts from this pain, and thus remaining motivated to contribute, might help drive adoption?
Oh no, I thought it was taboo to say such a thing!
Vaccines carry risk, but the risk of disease is orders of magnitude higher.
* Neither could chain of custody from manufacture to application.
* Subjects who experienced adverse effects were removed from the trial without further tracking.
* People with HSV-1 were administered a vaccine based on live HSV-2.
* Subjects were tracked with self-assessment questionnaires of poor design.
* Subjects socialized with investigators and with at least one cheerleading investor.
If this story was about a pharma company, we'd be calling for heads on spikes But there's a rooting interest for underdogs (especially those up against the US regulatory state).
These questions all have purposes, and are there for a reason. For example:
"That was the first ten pages or so of the Application. The rest was increasingly bizarre questions such as “Will any organs be removed from participants during this study?” (Look, I promise, I’m not a Nazi)."
There are a number of studies done by Not Nazis that still involve removing organs.
"And: “Will prisoners be used in the study?” (COME ON, I ALREADY SAID I WASN’T A NAZI)."
I'm not a Nazi, and for one of my studies, the answer was yes - prisoners get treated at the hospital I was using EHR records for, which meant they were in the data set. As prisoners have a very distorted capacity for informed consent, it's an important question to ask.
"And: “What will you do if a participant dies during this research?” (If somebody dies while I’m asking them whether they sometimes feel happy and then sad, I really can’t even promise so much as “not freaking out”, let alone any sort of dignified research procedure)."
This is a perfectly fair question to be asked during a study - especially as for many trials, determining whether or not the death was due to the study is an important question.
As far as I can tell, this essay is "In which Scott Alexander didn't do his basic homework, but still wants to be able to do a biomedical study, and is upset the IRB didn't pre-emptively read his mind and tailor his form for his particular case."
He actually made me feel sympathetic toward the IRB...which is an impressive feat in and of itself.
As the purpose of the IRB is to police studies, then it indeed has an inherent responsibility to work at actually understanding those studies - not to simply design one-size-fits-all forms and promulgate top-down rules as its primary requirements. Such behavior reeks of an entrenched institution that does little besides bureaucratically gatekeep to perpetuate its own power (and therefore is actually shirking its intended purpose!), a trend which is eating our society alive.
- Patients not knowing what they're being studied for
- Patients signing their consent forms using an alterable substance
- Study discontinuation criteria that aren't well defined
Are genuine problems with his protocol. They might be valid (though the last one really needs to be more specific), they might not be, but the process of asking questions is how the IRB comes to understand how a study is supposed to work, and what ethical issues are involved.
I've worked with IRBs in one form or another at a dozen different institutions now. Sometimes they're annoying, to be sure, but to be frank, reading his account, I doubt I would have approved his study.
And frankly, a lot of what he's put down suggests a really dangerous lack of understanding of medical ethics. He clearly didn't pay much attention during his mandatory training.
Hell, a lot of it suggests a really poor understanding of study design.
There does seem to be a few points where ethics do directly conflict with his desires (eg getting explicit consent to be "part of a study" versus priming the patient), but reasonable stakeholders should be able to mutually work out their real concerns, rather than go off the rails with arbitrary bureaucracy - say, ask the question first for the clinical purpose, and then obtain consent to incorporate that data in the study itself.
> - Patients signing their consent forms using an alterable substance
You're really referencing the signatures being in pencil?! I don't see how this could possibly be construed as a strict ethical requirement. Please do enlighten.
Furthermore, it doesn't rise to being an ethical problem unless fraud actually occurs. And I'd be more concerned about the possibility of signatures being straightforwardly forged by a researcher ("here, let me help you write your name") regardless of a pen being used.
There's a huge difference between research and routine practice, as well there should be. Note that things that are actually done for quality improvement and routine care are exempt from IRB regulations. Even at the most onerous institution I was at, that was a one page form. At the most lenient, it was a online "Choose your Own Adventure" flow chart.
He'd know that if he had paid attention at the training past "Don't be a Nazi". Because IRB training is pretty much universally run by CITI, and I've done that training more times than I care to think about.
"There does seem to be a few points where ethics do directly conflict with his desires (eg getting explicit consent to be "part of a study" versus priming the patient), but reasonable stakeholders should be able to mutually work out their real concerns, rather than go off the rails with arbitrary bureaucracy - say, ask the question first for the clinical purpose, and then obtain consent to incorporate that data in the study itself."
One of the things that he ignores is you have to justify these things, rather than just going "I want to do it". I've been part of studies that do not even ask for informed consent due to the study design, but that's a point of negotiation and discussion with the IRB. And I absolutely think their default stance should be "No".
"You're really referencing the signatures being in pencil?! I don't see how this could possibly be construed as a strict ethical requirement. Please do enlighten."
I am. Pencil is really, really easy to alter. Having a requirement like the form must be signed in ink is a reasonable compromise between something like pencil and, as you allude to, notarized and witnessed, etc.
And his excuse, that his patients can stab themselves with pens, seems untrue for a subset of pens (felt tips) and still true for pencils.
Basically, he cast himself in the best, most put upon light as possible. I am already pre-disposed to muttering about the IRB, because it does add time to things, and sometimes is silly. That I came off reading that going "Noooope!" is a really bad sign.
For example, the department had a huge amount of patient information stored in their regular computer system. But the IRB demands an archaic pen and paper system for storing patient records, rather than using the computer system that exists. You could argue that maybe the department computer system was bad, but in that case it should be fixed, rather than creating a parallel system.
Similarly with the issue of consent. The doctors were already giving questionnaires to the patients and already using them to determine what the treatment should be (counselling, medication, etc.) Why do the consent requirements suddenly become completely different when it's part of a study? Should the doctors be asking patients to fill out a consent form each time they give a questionnaire? Should the forms be in pen, even though mental patients can use pends to hurt themselves? I don't know, but I do know that having bizarrely different criteria for a study versus normal medical practice seems unwise.
I agree that it would be good to have better-defined criteria for stopping the study. It sounds like Scott had an informal goal of 100 data points, but didn't manage to hit this goal.
I'm sure IRBs do some good. But it's really hard to take the IRB's side in this particular case. They weren't even following their own rules (the person asking him to take ethics training hadn't taken the training herself-- which would have probably torpedoed his study even if he had done everything else right.)
The department's clinical activities and research activities are subject to different Federal regulations, and the IRB doesn't have any authority over standard clinical care.
"Similarly with the issue of consent. The doctors were already giving questionnaires to the patients and already using them to determine what the treatment should be (counselling, medication, etc.) Why do the consent requirements suddenly become completely different when it's part of a study?"
Because it's part of a study. An entirely different ethical framework applies when you are trying to treat someone vs. when you're trying to learn how to treat them. Sometimes those criteria are laxer (for example, you can use investigational protocols and things that aren't yet standard of care). Sometimes they're not.
"Should the doctors be asking patients to fill out a consent form each time they give a questionnaire? Should the forms be in pen, even though mental patients can use pends to hurt themselves?"
Again, different contexts. They happen to be in the same locations, but very different contexts. For example, your doctor can access your entire medical record vs. a hospital's EHR system for your care. That does not mean that the doctor should be able to access all your information, for all time, for any reason.
"I don't know, but I do know that having bizarrely different criteria for a study versus normal medical practice seems unwise."
Most IRB regulations exist for a reason, and no, study vs. normal medical practice are different, because while the risks may be the same, who bears the benefits are not. The entire purpose of the IRB is to ensure the risks to the study subjects are kept to a minimum because they are also unlikely to bear most of the benefits. Medical ethics works on the ethics of a single patient.
They are different by design, and through long experience.
"I agree that it would be good to have better-defined criteria for stopping the study. It sounds like Scott had an informal goal of 100 data points, but didn't manage to hit this goal."
I meant that it would be halted if the patients become "Violent". Violent is not a clear definition.
Though he also probably should have had power calculations in his protocol. IRBs really don't like studies that don't have a chance of producing information.
"I'm sure IRBs do some good. But it's really hard to take the IRB's side in this particular case. They weren't even following their own rules (the person asking him to take ethics training hadn't taken the training herself-- which would have probably torpedoed his study even if he had done everything else right.)"
The person asking the questions are not necessarily the people who make the decisions. For example, most of the questions at an IRB I worked with were asked by admins, because they've got dedicated time to do it, and can go through a checklist for things like "Is violence well defined?" or "You seem to have deviated from the normal way this goes..."
After that, it goes to a formal committee who take a lot of training, who either approve, disapprove, or ask other questions.
The concept is absurd. "Whoa guys! A resident just entered the room! Now we have to fill out all our forms in pen rather than pencil because the ethical standards just radically shifted! Now we're learning how to treat rather than treating!"
I think you are also missing one of the big points here, which is that the study protocol was exactly the same as what the doctors were doing anyway. So philosophical tangents like whether risks should be minimized more in research or treatment are irrelevant in this particular case. The risks are the same because the protocol is the same.
You can disagree that they should be entirely different ethical frameworks, but at the moment they are by Federal law.
And the risks might be the same, but the benefits may not be. And beyond that, it's the researcher's burden to show that, not the IRBs to take their word for it.
Injecting viruses into desperate people, on the other hand, has some pretty obvious risks. One can argue whether the current amount of regulation regarding it is excessive, but its pretty clear there needs to be some regulation.
Plus, note that the problems here weren't only regarding safety issues. The study was also poorly designed from the standpoint of getting meaningful data out of it. So even if one was indifferent to the safety of the study participants, the whole exercise seems to have been close to worthless, since the data is unusable.
Gosh I here that statement echoing in my head about the problems in my life...
Also, this is exactly the kind of problem in healthcare that is completely government and establishment driven. Physicians keep propping their monopoly up with literally hundreds of millions of dollars in lobbying/bribery.
Every profession in medical seems to be similar, dumping massive money into politicians:
https://www.opensecrets.org/lobby/top.php?indexType=s
Then their 'scientific method' has completely let me down after reading what these 'experts' consider a study. Lots of hypothesis getting validated by any-means-necessary. Not to mention their understanding of statistics usually tosses in the word 'P-value' for extra points, despite their population sample and study being a terrible representation of outcomes.
Medical is corrupt AF
That is a question, not a statement "Since I can help...". The problem is that he did significant harm to people trying to answer that question. The results of his experimentation are not fully realized, but the story indicates his injections caused harm in some cases and do not appear to have achieved the promise cure in any(?) of the cases.
The counterbalance question is "If my hypothesis is wrong and I harm people, isn't it my duty to do no harm?"
He missed out the 'first do no harm' bit. It's quite important.
>I will follow that system of regimen which, according to my ability and judgment, I consider for the benefit of my patients, and abstain from whatever is deleterious and mischievous.
Based on the reported specifics of the rogue vaccine trial, it would seem Halford did most of this. The article questions his judgement over the confusion between HSV-1 and HSV-2 strains, but that seems the most notable lapse. The picture painted is of a scientist trying to cure a disease in spite of obstacles - not disrupt the system or intentionally cause harm.
[1] https://www.health.harvard.edu/blog/first-do-no-harm-2015101...