Hacker News Reader

This. Having been through the FDA process myself (though we didn't make it to NDA, sob) I have to say that dealing with the Agency was great. Tedious, but great, and nothing seemed stupid except some trivialities at the margin (e.g. certain formatting issues).

Honestly the worst parts of dealing with them are direct consequences of people trying to game the system. We went out of our way to make sure nothing we did could give that impression and got excellent guidance and response.

Same.

Been through that process with a couple products from a medical imaging software startup. Extreme professionalism and competence was the lasting impression I got from my time interacting with them. We were making software that could be used in oncology for diagnosis, analysis, and treatment planning, so yeah, we got the regulatory proctology exam. But I wasn't annoyed at all. They were highly competent people who took their duty to protect the public from me seriously.

I took no offense at all.

Sounds interesting! Which startup?

>Do they get it wrong? Yes, occasionally. Not frequently. Occasionally. But that also, is science. The FDA follows rigorous statistical procedures, but even p<0.05 (or <0.001) still means false positives can happen.

The issue is that there is a huge lack of transparency. When a third of the studies involved in approval of anti-depressants were withheld from the public [0] and even some withheld at the insistence of the FDA [1], what good do low P-values do?

Hard to trust the science when you're not even allowed to review the papers for yourself.

[0]: https://www.nytimes.com/2008/01/17/health/17depress.html

[1]: http://www.washingtonpost.com/wp-dyn/articles/A9802-2004Sep9...

Well, current medical research is not really science, more of a sort of office politics with added prestige and financial incentive.

Actually you can read all the papers you want, you have no guarantee that what's on paper is what happened. Academic and industrial misconduct and epidemiological ignorance are so prevalent in the field that it's more akin to a religious cult.

No, unfortunately I'm not being pessimistic. This is also the case for the FDA people. They can't know, even by doing good work and to a very large extent, whether something will prove harmful or useful.

>you have no guarantee that what's on paper is what happened....

Can't speak for others, but we were deathly afraid of submitting anything dependent on papers that we had not verified ourselves to 100% satisfaction. This actually held us up a lot, but it was safer than taking a chance that you would get it by those people at the FDA. Now of course, we were making software, and when you deal with computer science research, there are a lot of BS papers out there. I remember reading through papers on ray tracing, which had supposedly been peer reviewed, but were presenting arithmetic that was entirely incorrect. Literally every paper we found on this particular algorithm was riddled with almost identical mathematical errors. It was crazy. (I would say that computer scientists have got to be the worst mathematicians I've seen during my time in the sciences.) In any case. I knew that one of the people we were dealing with was a pretty good math guy. Yeah, we decided not to chance it.

Having bogus research out there is one thing, getting lucky enough to slip it by the FDA is something entirely different. Don't mistake those two things for the same issue, it could cost your company. Or worse, if you do manage to slip it by the FDA, it could cost your freedom when your bass ackwards ortho normal basis calculation switches left and right, and the surgeon goes in after the wrong kidney.

During my MSc, I was implementing, validating, and comparing multiple numerical methods for a certain field of Aeroacoustics. Most (peer-reviewed!) papers had errors, some glaring, some subtle. The cynic in me thought those were done on purpose so that results were harder to reproduce.

There was one specific signal change (a + to a - in an equation) that took me almost a month of hitting my head against a wall to figure out...

Yeah, people need to learn that just because it's published doesn't mean it correct. The problem worsens when people just link papers they don't understand when arguing online.

Having worked closely with FDA, I've been very impressed by the people there and how thoughtful they are despite the pressure they're under and the almost complete lack of good software tooling due to the sensitivity, regulatory requirements, and complexity and specialization of the data. Most of the medical officers there could easily be a very successful (and much higher paid) private physician or a pharma researcher somewhere but choose to remain at FDA.

On one point people are making specifically -- unlinking safety with efficacy is really tough -- an immunooncology product that causes serious AEs/hospitalizations in a quarter of the population and costs $250k and only extends life by 6 months may be approved because the alternative is almost certain mortality; whereas another drug may not be approved simply because it increases bleeding risk in a small percentage of people because there are good alternatives already.

With respect to cancer drugs, while generally no one can be prescribed medication that has not passed clinical trials, in the case of terminal patients who could potentially be saved by a drug currently in trials, there is a system for allowing them access to the medication.

It's called "Expanded Access" (also known as "Compassionate Use") https://www.fda.gov/news-events/public-health-focus/expanded...

The other reason why data sharing is hard is because it costs on average over $1,000,000,000 (1 billion USD) and 10 years of time to whittle over 10,000 drug candidates into a single new FDA approved NME (new molecular entity).

If you spent a billion dollars and a decade on a project, you will consider the data you collected to be a trade secret and would protect it with everything you can, because you have to now take this drug to market and make up the billion you spent making this drug. If you simply shared all your results, foreign competitors will simply copy your work without paying you, and will then offer your product without the cost of R&D and regulation built in: AKA they will charge 1/10 or less than you and you will struggle to profit anywhere they are in the market.

> If you spent a billion dollars and a decade on a project, you will consider the data you collected to be a trade secret and would protect it with everything you can, because you have to now take this drug to market and make up the billion you spent making this drug.

You would also be heavily incentivized to make sure the numbers show that your drug is effective, with very limited side effects..

You can try to fudge your data but the FDA is the most rigorous science based regulator in America and possibly the world.

And, the FDA monitoring of drugs requires more than Phase 1-3 clinical trials (which cost hundreds of millions of dollars to produce some of the most rigorous science in human history), but they also do post-release data analysis (so called Phase 4), and have even recently pulled drugs which are not meeting the Phase 2 and 3 standards for safety and efficacy over existing treatments (not efficacy over placebo, the results must show efficacy also over existing drugs).

Not to mention, the FDA fines of pharma corporations are some of the largest fines in American history.

So with the FDA you have

* A regulator using science forcing big business to spend 10 years creating scientific evidence that their product is safe and effective. No other industry on the planet has these requirements.

* A regulator using market data to actively govern products and pull products which aren't safe and effect (pretty normal)

* A regulator assessing multi-billion dollar fines pretty regularly https://en.wikipedia.org/wiki/List_of_largest_pharmaceutical...

You might find this surprising but pharmaceuticals is a money losing industry in the long term. Most of these companies are looking pretty bad, they're spinning off any division which could potentially be profitable (biologics, medical devices maybe) but your average business of finding new drugs? It's not profitable, the regulation is so intense and the work is so hard and competition which ignores intellectual property is so damaging that it's very likely most major pharma companies will go bankrupt in the coming decades.

When you compare how much science the FDA makes drug companies do to any other industry, food, finance, supplements, makeup, cars, houses, anything at all... nothing else compares. A forensic audit is nothing compared to a Phase 3 clinical trial. Far less goes into proving a car is safe than a drug. Etc.

The FDA is doing incredible scientific work with an extremely thin budget.

This is a good argument for government funded R&D, and government owned drugs.

They cost a ton of money and take forever to create, so the product becomes very expensive, and rights to it are owned by the creating company. The government empowers a body to regulate the flow of new drugs on the market, and once the drug meets regulatory requirements, the government - in many cases - pays on behalf of the patient.

A company is interested in making a profit on their time investment, whereas the government could be content to break even (for the good of their citizens).

If the government instead paid a company to do the R&D, maybe even providing access to approved facilities where the work is to be done, the government could then retain ownership of that product and then create agreements with drug manufacturers to produce those drugs.

This solves a couple of problems, like sole producers of a product arbitrarily increasing prices (it is expensive and time consuming to set up production for a new drug, even if it is off patent), as well as supply issues (I don't think right now the FDA has any rules saying a drug company must be able to supply a certain amount of a drug to get their license to distribute).

A good argument against government funded R&D is that several of our modern medical challenges have been deeply embroiled in political debates.

AIDS/HIV(originaly called GRID) was originally considered a punishment for gay men who were seen in the 80's as being immoral and sinful by the establishment. Government-funded R&D for AIDS drugs would have been unlikely for decades.

Another example is hormonal birth control. Birth control in general is a huge political football with portions of the establishment wanting no access to birth control at all. If R&D were government funded it's possible for these people to prevent access simply by making funding of certain lines of research illegal.

I think you have to look beyond the price of treatment to really see the benefit of a healthy private enterprise here. There are still a ton of ways that government contributes to new treatments, like grant funding for basic research and development. But I can see your proposed future becoming a dystopia, where whoever is in power in government makes it illegal to sell certain patented drugs to their particular underclass of the week.

Thank you for this. So often we rip on government agencies from our perch on...Hacker News, when we really have no idea what's going on.

We do work with the Government in a different context, and I've found nothing but professionalism and deep expertise. In fact the people I've met in the agencies we're working with are some of the smartest I've encountered in my entire career.

>often we rip on government agencies from our perch on...Hacker News, when we really have no idea what's going on.

In a single department, we don't know.

As a whole, it's been shown over and over that like any organization, government becomes corrupt.

Between defense and universities, I've heard depressing ancedotes as well.

It's lucky and temporary you had a good experience.

That seems like a pretty cynical perspective to me. Like if Edison had said "it's been shown over and over that electric light bulbs don't work" after the first couple hundred experiments didn't pan out.

In every country, in every world, in every decade of humanity, in every department.

But this time will be different.

Very interesting take.

You clearly have never dealt with people from local jurisdictions, such as from the local city government or county governments, and even up to the state.

I hate to say it, but every one of these folks I have encountered falls into two categories: 1. Recent college graduates who don't know much if anything, 2. People who have worked a government job for a long time but have little intellectual capacity to understand complex subjects.

I deal with people in local, city, county, and state governments all the time.

Haven't met many recent college graduates, as they don't usually have the experience to land a government job. Have met a few old-timers. But otherwise almost all of them were efficient, helpful, and good at their jobs. In the specialized departments, they definitely had the intellectual capacity to understand complex subjects and could have earned significantly more money in private practice if they wanted to waste their lives in a rat race.

I guess I need to find a job in your area then, because my experiences here are the pits.

Also it’s important to understand the context that medicine wasn’t originally “scientific “ for all the reasons you are touching on.

Historically treatments were based on some theory of how the body works, and a plausible intervention either based job that model, or based on tradition and anecdotal experience.

As people tried to implement more empiricism it hits all the difficulties you mention and others ... it’s an extremely difficult ugly challenge.

We should have lots of medical reversals. It means we are slowly getting better. Probably things could move faster if there was some giant anonymized database of every patient treatment, with documentation of symptoms, intervention, and outcome.

There are so many reasons why that is impractical, I’m astonished we are making the progress we are.

> are completely ignorant of history and basic facts

Sam Peltzman's "Regulation of Pharmaceutical Innovation" does an evaluation of this based on statistics, and concludes that the efficacy requirements are a net negative.

It's not about the FDA scientists being incompetent. It's about the increase in cost is so large that far fewer new treatments are even attempted. Conditions that affect a smaller number of people are simply not worth finding a treatment for.

>Data sharing is very very hard given the sensitivity of the information (patient data) - making it effectively impossible to have an open data exchange. (This is also due to the billions of dollars at stake on the pharma side).

There is promising work going on in this area to allow the sharing of models while maintaining privacy. (Differential privacy, homomorphic encryption, multi-party computation, etc.)

It's always worthwhile to learn the history of an organisation. That tells you a lot about it's culture.

https://en.wikipedia.org/wiki/History_of_the_Food_and_Drug_A...

This is true, but there may be moderate reforms that would work. Could drugs approved by European regulators be allowed without FDA approval? Which regulators do a better job?

> Could drugs approved by European regulators be allowed without FDA approval?

Why should they be? It's not like European regulators have a better track record than the FDA here.

As long as they're not worse, it could result in more approved drugs.

Well, Propranolol was available in Europe for its main indication about a decade earlier than it was allowed in the US saving 10,000s of lives. European drug regulation encourages competition in ways that provide for about a dozen EpiPen equivalents rather than just one provider that can jack up its price to $1000. You can buy sunscreen that blocks both UVA and UVB in Europe, though the FDA is thinking about maybe allowing those sunscreens in the US soon. They did miss Thalidomide, though.

> European drug regulation encourages competition in ways that provide for about a dozen EpiPen equivalents rather than just one provider that can jack up its price to $1000.

This has literally nothing to do with FDA approval of NMEs.

No, but it is a reason you might want to be able to buy drugs imported from Europe.

> No, but it is a reason you might want to be able to buy drugs imported from Europe.

Which is illegal in itself, for reasons that have nothing to do with FDA approval or the issues relevant to this article.