FDA and NIH let clinical trial sponsors keep results secret and break the law | Hacker News Reader

FDA and NIH let clinical trial sponsors keep results secret and break the law

sciencemag.org

289 pointsbarry-cotter63 comments

I just listened to the book Bad Pharma, by Ben Goldacre, which deals with this problem. The author works as a doctor and so has a bit of an inside view of how the industry does 'science' and how this affects doctors and patients.

The lack of published negative results isn't a problem unique to pharma but the drive for profits and the actual life and death outcomes makes it particular severe in this area of science, I think.

I highly recommend the book, it was both hilarious and terrifying.

Don’t forget to support Ben’s campaign to change this. https://www.alltrials.net/

There is also an opportunity for public databases of patient outcomes (post marketing) that we would all greatly benefit from. A lot of that data is still discussed behind closed doors and authorities are also to blame because it is their prerogative to force data to be open if they wish to.

We are working on it, but financing seems to be tricky for such an endeavor.

Sounds like a FUD magnet but I agree there would be major upside. Edit: To clarify, publishing individual patient outcomes publicly could pretty easily be spun as a privacy risk to trial participants and/or have a chilling effect on enrollment.

I’m personally invested in seeing something like this happen but if not approached defensively it could be vulnerable to getting shot down and made an example of.

I must note that according to the linked article, "pharma’s record has been markedly better than that of academia and the federal government". It's academia that is not reporting results, not industry.

That's an illogical conclusion.

Both don't always report results. Academia is worse. That doesn't make the industry good in any way.

I don’t know what problem that would solve. Academia rarely does trials that end up killing people. Unlike the whole CAR-T cell therapy. Companies like Juno killed almost 50% of their patient, before the FDA halted their trial, which was allowed to resume later, and kill more people. Granted they where going to die anyway. Still the like of Hans Bishop who us held as a hero in the industry because he sold his company for billions, while the product will never be approved. Even if were to be, it would sit on a shelf because it’s too expensive. So much for transparency...

(Edited some typos)

> while the product will never be approved

I must point out that some CAR-T cell therapy is already approved. I am not sure about Juno's particular candidate, but it isn't obviously insane that it was sold for billions.

I’m standing right now near the window on the 20th floor of a cancer center, typing this on my phone while my wife sleeps 10 feet from me.

This is high stakes medicine, yet the current environment has the docs on this floor terrified of doing anything that isn’t in the standard of care runbook. My wife has had elevated blood glucose levels for six months, which is not good when you’re fighting cancer, yet i can’t get anyone to treat it because there’s no clinical evidence that managing glucose levels for her specific type of cancer has any benefit.

Guess what. Nobody’s doing that trial any time soon. Meanwhile there are thousands of diabetics managing their blood glucose while fighting cancer and there is endless material available demonstrating that cancer cells thrive in hyperglycemic environments.

So, i don’t know anything about Juno, they could be sociopaths. But i have no issues with high risk attempts as long as the patients are informed and there are ethical standards for transparency. At least they are trying.

I sympathise with your wife's predicament. Good or sub-par treatment, doctors still get paid.

If you're not opposed to eschewing professional recommendations to not complicate treatment, there are two things that may be worth looking into:

1). Metformin: Helps control blood glucose by significantly increasing sensitivity of insulin receptors, i.e more glucose is shuttled out of the blood stream per insulin molecule. Not scheduled and very cheap. Also trials are being done on it for cancer prevention -- exactly because of Warburg's phenomenon (cancer cells and glucose)

2). Ketosis: Hit or miss. By abstaining from carbohydrate consumption, the result should be drastically reduced blood glucose levels. However, in some people (e.g type 2 diabetics) it may elevate blood glucose due to metabolic disfunction. In any case, caloric restriction in general have the effect of both increasing insulin sensitivity and decreasing blood glucose -- regardless of macronutrient profile

The doctor though should be able to provide access to a clinical dietician to help think plan the diet and think through the implications. I imagine some chemo and targeted drugs could interfere with lipid metabolism and there is also a risk of acidosis. So I would ask the doctor. It will probably be a case of "we don't know" but diet is basically the one variable under your control besides listening to the doctor.

There's one paper on it from a completed clinical trial. https://www.ncbi.nlm.nih.gov/pubmed/27525031?dopt=Abstract

Also clinical trials using metaformin during cancer treatment. You might consider enrolling in one of these if it is feasible.

https://www.cancer.gov/about-cancer/treatment/clinical-trial...

Awesome suggestions thank you. I had to go to Care Oncology to get a metformin prescription which just arrived at the house today.

I tried going the A1C route to justify intervention but anemia and transfusions both push A1C low. I also just got a prescription for a Freestyle Libre to monitor blood glucose.

We’ve never got quite to the level of ketogenic but have tried focusing on fats and proteins for calories and keeping carbs between 50-100/day.

Of course it makes industry better than academia for this particular case. Do not think in terms of good or bad, think in terms of better or worse.

The industry has a near-perfect report rate. Only one company missed a report (for a total of 1 missing result out of 629 trials from industry), and half of them never even reported anything late. Take out the worst two offenders, and they do better than literally every single academic and government entity. In terms of average delay, only two academic/government entities do better than any industry entity (save the two outliers).

I'm not going to claim that industry is perfect. But take a look at the graphs. It is a picture of an industry that is generally in compliance (enforcement could perhaps be stepped up), compared to academics and government who appear to believe that compliance does not apply to them.

How many studies from academia are actually industry-funded studies by proxy? Given all the pharma-funding of academia, I suspect the answer is non-zero.

If they were industry funded, then they would fall under “industry” in terms of reporting.

These are NIH grants to academia - industry is not involved.

How often are private companies paying at least in part for this research... also, since academia is abusing the patent system as much as everyone else, it's kind of expected.

The article states "based on data collected from the TrialsTracker website of the University of Oxford", which is Ben Goldacre's project.

I'd also recommend his TED talks which are really amusing and educating.

https://www.ted.com/talks/ben_goldacre_battling_bad_science

https://www.ted.com/talks/ben_goldacre_what_doctors_don_t_kn...

I don't care about whether they report results or if they miss deadlines for reporting. That's all this article is talking about.

But only studies that are pre-registered should be considered when approving a drug, treatment, or device, and any incomplete studies (that is pre-registered but not reported for any reasons) should be treated as "all the patients died", and all individual researchers on the preregistration should be appropriately tainted for future consideration of studies in which they participate.

The related story in the article, "Gaming the system" does report on this:

> .. a recent study ... examined 19 drugs approved by the FDA in 2015. In only 10 cases were all the trials provided to FDA for its decisions registered on ClinicalTrials.gov.

This is unacceptable.

All of the punishments you suggested and also any involved patents released to the public domain since the data wasn't.

Hiding research results should be seen as this scandalous, shameful, and fundamentally anti-scientific practice. A heavy bias towards business profits, which is exactly what this is, goes against objective science.

What if you kept repeating experiments, and only published when you got the results you wanted? That goes against my personal ethics and my understanding of how science is supposed to work.

> A heavy bias towards business profits, which is exactly what this is, goes against objective science.

Except the data shows that it's not the industry who's failing to report results, it's academia and the government who's failing.

When people like Michael Taylor run the FDA, that's clearly agency capture. The industry has infiltrated government positions, I don't know how anybody can act like they're innocent in all this.

> What if you kept repeating experiments, and only published when you got the results you wanted?

This is pretty much how it's currently done. Science is far less exploratory than you make it seem. When it comes to publishing you should think of it more like law. You make the strongest possible case for your conclusion.

This is very backwards. In science, the evidence for a claim is precisely how hard you tried to disprove it. This doesn't mean that you're wrong, of course, but it's a massive problem.

That’s only a subset of experiments, the ideal experiment makes any result publishable. For example the search for a new particle, don’t find it that’s fine. Alternatively, any result that’s in doubt like salt’s impact on cardiovascular heath.

Even better is really cheap experiments like analysis of existing data. That you can keep trying until you find something, sadly this means publishing datasets is a bad idea.

Especially since then you can't compare to prior trial data to newer trial data to see how they were crafting the selection process, which could derive hints for contraindications and the like - or simply to know if some medication is poorly distributed, in the sense of making sure a person fits the exact criteria, then its side-effects and damages will be far greater than what was approved at the end.

Hiding research results should always be treated as though the results were very negative. There are many reasons that a study might get cancelled that are unrelated to business interests (like breaking of blinds or various other methodological reasons why a study's data is corrupted), and it's okay(ish) to not publish these results -- there are no good protocols for publishing data about a study's failure, but it would be nice to develop those. But without knowing the reason for the failure to publish the study, it should be assumed that the results provide negative evidence for all hypotheses.

Pre-registration is supposed to solve this problem by listing the studies before they are conducted so that it is impossible to completely elide them. A drug approval would then look at all studies pre-registered for the treatment, and only at those studies pre-registered for the treatment -- whether those studies were complete or not.

Perhaps unbeknownst to some folks, the FDA as a regulatory body is funded in large part now by Congress but by GDUFA and PDUFA which allows the agency to collect user fees for both brand and generic drugs from the drug makers themselves, in essence being funded by the industry it is meant to regulate. So it is then understandable why a lot of the guidelines become flexible (note that the FDA cannot enact laws, that is the duty of Congress)

This is a very common complaint, but the other side of the coin is that FDA needs resources to do its job and any time the number of submitted drugs changes the FDA would need less/more funding from congress if it was funded only by congress. Collecting fees means the size of FDA's funding is correlated with the amount of work it has to do. The proper way to determine whether FDA is unduly influenced by this is to look at the actual decisions it makes, which is something I remember reading medical researchers already tried to do.

GDUFA guarantees a decision within 10 weeks, which is considerably better than the running average before and even extreme cases such as 13 years before addressing applications.

There are unfortunately many cases that have been decided on the basis of finances rather than clinical justifications. I don’t envy the agency, stuck between the disdain of the public, pressure from industry, and continually having to justify their existence to Congressional committees.

This is one place where I would like to see an immutable blockchain database used, as a custody of records for decisions made.

> Few trial sponsors have consistently done so, even after a 2007 law made posting mandatory for many trials registered in the database.

Then the reality is, it wasn't mandatory.

It's not reasonable to expect citizens to trust these entities when their conduct is not trustworthy.

If there ever has been a sector where mandatory full transparency is useful then medicine would be a good candidate. The outcome of trials could be very useful for other researchers.

Same should apply for pricing. The fact that prices are completely opaque in the US makes it hard for patients to make good decisions.

I'm kind of shocked that logging these results isn't already common practice.

I think this is the primary purpose of the FDA (though not the NIH). Providing a shield from transparency, and even against criminal liability, is just another part of regulatory capture.

Coincidentally, I've just heard Milton Friedman talk about "Health Care in a Free Market". Fascinating and eloquent.

He argues that the FDA did more harm than good, and that society would be better off without a governmental supervision for health care in general.

-- https://www.youtube.com/watch?v=-6t-R3pWrRw

btw, in the end he makes a strong argument for negative income tax. This surprised me, as my model of his world-view was apparently more simplistic that it ought to be.

I disagree. FDA have set guidelines for clinical trials and moral standards to it. For an example you cannot experiment on a person without their consent and you have to state out what your trials is doing. It all lay out. It's because of Tuskegee syphilis experiment that we have IRB and FDA.

FDA also love stating that they prevented a drug that causes birth defect a while back unlike Britain.

Another example of government regulation working is cars. Death relating to car crashes have fallen every year with safety regulations.

It's not perfect but there are many cases that it works and there are laws and rules set in place.

The free market is not the panacea for everything.

The Tuskegee syphilis experiment was run by US Public Health Service which is currently part of the HHS. The FDA was founded in 1906, although it has earlier origins, which is well before the study was begun.

Falling car crash rates don't prove regulation is successful. Consumers shop for safety too. It's difficult to compare the counterfactual, but to make this claim, you need to at least compare the rate of change of crashes in unregulated countries or times to regulated countries or times.

For both auto safety and pharmaceutical safety there is a large degree of consumer information disparity, information unavailability, and disincentive for the individual to protect the public interest. It takes a lot of resources to test those products, and consumers don't really have the resources to do it themselves. In some cases, insurers could possibly foot the bill (eg. IIHS), but that model still has a couple of big weaknesses: consumers may not be informed enough to parse the results, consumers might prioritize short-term needs over public health, and an insurance body may still have some incentives working against public benefit.

Things like truck under-ride guards likely wouldn't be on commercial trucks if they weren't regulated. Consumers aren't going to all voluntarily coordinate to buy vehicles which have crash-compatible bumper heights. Antibiotic misuse would be even more out of control if it weren't regulated.

Mexico is a good example for road regulations. They do not have the same crash regulations as the US, and many old model vehicles are still legal for sale there. Things like the 1950's VW Beetle were still sold there into the 2000s. They have the highest death rate per distance traveled in the world.

Ah yes. This is why, before we had the FDA, everything was fine, and why nobody saw the need for the FDA in the first place </s>

The Libertarian capacity for ignoring Chesterton's Fence is nothing short of astonishing. Free markets require free information - the FDA was created in large part to enforce accurate labeling.

https://www.fdareview.org/issues/theory-evidence-and-example...

It turns out we can directly compare pre-1962 FDA with the current FDA and find strong harms outweighing any benefit of the current regime.

If all the FDA did was "enforce accurate labelling", they wouldn't be responsible for hundreds of thousands of deaths from delayed approval of drugs. Maybe that's not the part people are complaining about.

That might be a valid point if the parent comment had been similarly nuanced. But it said "no government supervision", not "roll back to 1962".

I find this kind of motte-and-bailey stuff ubiquitous in discussions with libertarians, as the "philosophy" amounts to anarchy which most people realize is bad.

Note that I'm not the one who posted the parent comment.

Anyway, the Milton Friedman talk linked explicitly mentions the same beta-blockers example as my link discusses and cites it as causing tens of thousands of deaths. He says it's been a disaster over "the last 20 years" - the talk was in 1978, so this is roughly the 1962-1978 period. And the "governmental supervision for health care" mentioned is broader than the FDA.

You don't talk to libertarians much, do you? I don't know many libertarians that would argue that accurate labelling shouldn't be enforced by the law.

Drugs quality testing could also conceivably be performed by a non-government regulatory agency (e.g. Underwriters Labs)

"People weren't dying from bad drugs before we had the FDA. I mean, it just didn't happen."

--Ron Paul, noted libertarian and advocate for the abolishment of the FDA, who is apparently ironically ignorant of the phenomenon of "snake oil".

I think you've proved my point.

As a recipient of free healthcare, i would never ever like to see "free market" near my "paid by government" healthcare.

So what you're saying is because FDA isn't enforcing rules strictly enough we should leave it to free markets and have no rules at all?

Libertarian logic I guess...

Consumers unload responsibilities onto regulatory bodies. When the regulators function appropriately, the result is better than no regulation. When regulatory agencies are captured by industry and better serve commercial rather than citizen interests, it is unlikely that citizens will recognize that their government is no longer working for them. At this point it may be better for consumers to be forced to pay more attention to the product.

And this creates multiple points of competition, unlike regulation which is somewhat anticompetitive. Transparency, efficacy, cost, reviews in the age of the internet, all combine as a form of emergent regulation which may outperform explicit agencies in some cases.

If the state of online reviews is any indication, things would devolve into a clusterfuck of disinformation.

Committees Of Concerned Citizens On The State Of Toothpaste will pop up and dissolve like mushrooms, some of them funded by this or that corporate interest, some being incompetent to evaluate state of anything, and the rest drowned in all the noise.

Emergent anything only works if there is an explicit external force pushing things towards a specific organization or a set of laws of nature that produce outcome you happen to find satisfactory.

I guess your point was that current state is worse than that already, and that's a valid point, although I don't think this is the case.

This would be a nightmare, the average consumer has absolutely no capacity to make informed judgments on their own healthcare, they would get eaten alive, and historically they have been.

The general consensus of many workers in the industry is that the FDA's regulation is about right, rather than being either too tight or too lenient.

It is willful or reckless ignorance to believe that the absence of the FDA will not have serious consequences. There already exists a segment of the drug industry that is nearly completely unregulated by the FDA, as a matter of law. I am of course speaking of the health or dietary supplements. And the experience here does not bode well: there are a sheer number of "miracle" pills and cures that come out of it, unhelpfully promoted by snake oil salesmen like Dr. Oz. And investigations have found that many companies can't even be bothered to put the active ingredients into their products.

There's another area that's unregulated - off-label uses. See https://www.fdareview.org/issues/theory-evidence-and-example... for a discussion of the evidence from off-label uses of approved drugs.

From my own experience with the Canadian health system, mentioning it as basically a single payer system, not allowing off-label use causes problems - at least the lack of and ability to initiate in a cascading process to initiate research for off-label use is a problem.

It can be argued both ways. Another option without FDA would be people will self-regulate: determine who they trust - individuals, organizations, or networks - and who they don't trust based on hearing word of mouth experiences being spread vs. people being manipulated via bombardment of shallow, cheap advertisements - allowing a functional mechanism of Darwinism while preventing regulatory capture and squashing of drugs that were known to provide great benefit - but couldn't be patented so were controlled - like marijuana, MDMA, psychocybin, etc; presumably anything that kept in circulation in the black market was providing individuals with benefit, and arguably things like opiates kept and pumped into the public market did far more harm than even perhaps the total of harm caused by the black market drugs.

Edit: Just to clarify, I'm not arguing for or against here, I just stated another option - so thanks for that downvote whoever the lazy person was did it.

Naomi Klein on Milton Friedman: https://www.theguardian.com/commentisfree/cifamerica/2010/ma...

Friedman and the "Chicago Boys" advised Augusto Pinochet in Chile and used his dicatorial powers to ram through changes into the Chilean economy. I don't think Friedman is someone to admire.

Welcome to GoopTV!

The pharmaceutical industry is full of unscrupulous bastards, news at 11.

The FDA operates as a corrupt organization and, if wasn’t a government agency, would be prosecuted under RICO.

Reply on news.ycombinator.com