The F.D.A.’s Medical Device Problem
nytimes.com
nytimes.com
I can see the author's point of view, but this is a rare case (for me personally) that I am not in support of more government oversight.
[0] http://www.medtronicdiabetes.com/treatment-and-products/mini...
http://www.motherjones.com/kevin-drum/2013/10/heres-why-your...
The statistical result of the 1962 Amendments is that the beneficial effects of reducing the amount of bad drugs being released is far more outweighed by losing the beneficial effects of drugs being delayed or not being developed. But people only see the former.
This seems to be a trend, when people know about a certain field well, they are against the government oversight of it. The issue of course is that people are usually intimately familiar with maybe ONE field, but get an opinion on everything, and thus are very cautious of all the things they don't know anything about.
Software people understand the problems with "making encryption illegal", regular people don't and are more willing to hear the high level "safety issues" of letting hackers hide their information.
You are well aware of the issues with these devices, most people aren't and are more willing to hear the high level "safety issues" of letting any ol' device get into the hands of consumers.
Scientists know about all the potential benefits of stem cell research, most people without biology degrees are more willing to hear the high level "safety issues" of the frankenstein stories we're sold.
Because we have the "nutritional supplement" market as a working example of "We give people stuff without bothering to see if any of it actually works".
Or, even, if its actually the stuff that it claims to be, much less whether the stuff it claims to be works.
[0] see, e.g., http://well.blogs.nytimes.com/2015/02/03/new-york-attorney-g...
What pharmaceutical company is going to bother getting FDA approval for an already-patented substance that is dirt cheap and readily available?
In the EU, melatonin is regulated and requires a prescription -- which seems to me to be an unnecessary cost and hassle.
Presumably they are making a profit at it. They spent a bunch of money to set up factories to produce, package, and market melatonin, why suppose that FDA approval, but not these hurdles, would stop them?
Or what about the patent-holder (if there is one)? One would think they would have ample incentive to seek FDA approval if they have a temporary monopoly on producing the compound.
Clearly more hurdles are... more hurdles?
I guess I just don't see how this question isn't a tautology. No one is arguing that the other hurdles couldn't stop it either. But if the manufacturing costs are $1 million, and the FDA costs are $1 million, then I don't see how it isn't obvious that needing BOTH is $2 million and thus MUCH more difficult. I agree that if we could magically reduce manufacturing costs to zero it would be even better. But your argument literally seems to be "since this is already expensive and difficult, we can make it arbitrarily more expensive and difficult with no reduction in success rate". Which obviously isn't true. More to the point: we know that it can get through the manufacturing hurdles because it did. We thus know this company is capable of 1x costs. Had they ALSO had additional costs from the FDA then it would have been >1x costs and thus we don't know if they would have succeeded.
Perhaps an analogy. Let's say someone proposed a new fee for graduating college of $50,000. I would say "well, that might make it so less people can graduate, since it'll cost more". Then you might respond "But what makes you think THIS cost will prevent them, and not the existing expensive tuition?". Well, both do, but they already pay one and we're in control of a theoretical second one.
> Or, perhaps more to the point, why this particular hurdle is not worth the supposed consumer protection benefits of FDA regulation?
This on the other hand is a completely reasonable question, but not more to the point, just a a compeletely different point than "the company already has costs, what is the problem with more costs?"
Presumably they would not if they had to fund an arduous regulatory adventure while competing in a market against unregulated peers.
I think the best we could hope for here is that the USDA or some independent lab might test & certify that a bottle of melatonin contains what it says it contains, without respect to the safety or efficacy of melatonin.
There are two dangers: letting a bad device into use, and preventing a good device from entering use. And both "good" and "bad" are relative here, so it's a tricky dance. Personally, I thought the FDA was doing about right. They forced us to really have all our ducks in a row, but that's actually appropriate.
But with all that said, I personally found it a horrible environment for an engineer to try to live in, and I got out.
Its interesting that SV is often criticized for not taking on these "big important problems", but at the same time these are markets that can be impenetrable given that the entire fate of the company can be in limbo for years during approval, requiring tremendous up front investment that is harder to predict.
True. Management mentioned it from time to time, as a barrier to entry for competitors.
I worked for a company that made an FDA approved diagnostic device. And to be clear, for the condition it targeted, an incorrect diagnosis would kill. The approval was slow, expensive, and cumbersome. I absolutely support it. Consider the average code quality you see with every piece of software you touch. Hell, even apple can't update their iphones -- with very few models to test -- without killing wireless on some of them. The FDA required that your code was tested so that given certain inputs, it produced correct outputs every single time, and documentation to that effect.
I wish we could speed the process up, but I think it's the best alternative we have going.
edit: this was my first serious dev job. They didn't use source control. "source control" was a random computer holding the master copy; after a month or two of development, you'd spend a couple days merging your code into the master copy then copy the merged codebase to your desktop. All 8 devs did this. There were wild merge problems, as you can imagine with that much divergence. "History tracking" was copying the entire directory structure to a series of labeled directories.
There were two ways to license the machines: one for human testing (with fda approval), and one for non-human testing. The latter got releases every 3-6 mos. One at least one occasion, code was shipped so broken that it conclusively proved it had never been successfully run, not even once. The "deploy" process was to copy code from the master server, hit build in visual studio, and burn that to cds.
Automated tests? What are those? Test suites were run instructions written in a spiral notebook. (To be fair, it's hard to work with bio samples.)
I wrote a bunch of code that basically mocked the device by installed a special driver that replayed saved data files. I wrote this on my own time so I could occasionally work from home without lugging a giant contaminated machine back and forth. I shared it with coworkers and it eventually made its way to the testing lab on the other side of the country without my knowledge. When I quit, I deleted all source code. It turns out they hadn't kept a copy of this driver and frantically contacted me several months after I left when an embedded system upgrade broke it.
Many of these are symptoms of a company run by bio scientists who viewed engineering as a cost center and barely even a real career. There are lots of companies like this.
But that's not always the case, it's not even the average case. Where there is much lower risk, there should be much lower barrier to entry. Treating every medical device as if it were a likely Therac-25 is not even remotely a good solution, and in some of these cases the lack of approved medical equipment is directly contributing to many peoples' untimely deaths, and to even more peoples' poor-health and/or poor quality of life.
The result is that sometimes you end up releasing a suboptimal product, and NOW YOU CAN'T IMPROVE IT. Having some sort of intermediate level of testing/approval seems like it would be incredibly beneficial.
That said, I'm obviously biased given all the frustrations I've been running into at work.
Another thing that's super annoying is that the guidance document for weather or not a change requires a new 510k is woefully out of date and hilariously vague. It's from 1994 I think (so it doesn't deal with the concept of software very well), and the last time they tried to update it, they made it so specific that it got shouted down while in draft form.
The specific issue here is that the vagueness and lack of guidance severely penalizes small players. Larger players both have the experience and connections to navigate the rules, and also have the financial resources to absorb the losses when they gamble and lose.
I honestly think that this is a net lose for everyone, both in terms of health outcomes, and total cost.
Like you say, deciding whether not change X is a memo to file, a special, abbreviated, or full 510(k) is pretty much black magic because of how vague FDA is, so it basically comes down what your Regulatory VP is willing to live with, politically, inside the company.
"The specific issue here is that the vagueness and lack of guidance severely penalizes small players. Larger players both have the experience and connections to navigate the rules, and also have the financial resources to absorb the losses when they gamble and lose."
I think the latter part of your larger players argument is most valid: it's not so much that the big guys have people in FDA who can give them informal reads, though that's true. It's that the big guys can take the hit of another year of delay when something doesn't go their way.
"This act would also create a new, faster approval process for “breakthrough technologies” that are believed — but not necessarily proved — to offer significant advantages over existing alternatives. This would allow a device to be approved based on even lower standards of evidence than are currently used, on the theory that the need outweighs the risk. The legislation defines “breakthrough” loosely, creating a perverse incentive for manufacturers to use this term both to take advantage of the faster approval process and as a marketing gimmick."
"Breakthrough therapy" designation already exists for drugs. The FDA identifies those drugs that have shown promising results in early trials for diseases where there are few treatments.
The reason this is being done is because the choice isn't between an untested treatment and a tested treatment, it's a choice between an untested treatment and death. Why shouldn't physicians/patients be given an opportunity to try out a therapy when no alternatives exist?
Obviously (and the FDA already does this), the risk/benefit should be carefully analyzed. It makes no sense at all to allow a device on the market with little data if it doesn't provide a substantial benefit where none currently exists.
Concerning the follow-up trial that the FDA requires of manufacturers: I'd have no problem at all if the FDA provided "temporary approvals". You're device/drug looks promising, so we'll approve it now, but you need to run a full trial to provide the needed data. That trial will take 2 years to conduct and readout? Great, you now have a 2-year temporary approval and if the data doesn't show up, we pull your device/drug off the market.
It's an op-ed. The entire point is that it's biased.
This is farther than I would have expected the FDA to go. I can see releasing a device without full clinical trials, because those can be extremely difficult to set up and can create very long delays. Imagine having to find a large enough number of people with just the right disease state and who consent to be part of the group, then getting the right control groups, building relationships with labs and hospitals in the right places, etc.
But shifting reponsibility from the FDA to a third party is something I'm not sure I'm comfortable with. There's lots of precedence: nobody ships their devices off to the FCC for emissions testing, there are independent labs that sign off on that. Likewise companies such as UL that set safety requirements, have been around for decades. But as much as we bitch about the hoops we have to jump through to keep the FDA happy, I think deep down we're all glad they are there to do the job and keep us honest.
The moderating influence is that device manufacturers as a group are generally extremely, repeat extremely, conservative. The Quality Assurance engineers are generally a brick wall that have to be completely appeased before they will allow anything to be released.
This is exactly what happens in Europe, though. And it's one of the reasons we generally release products there first.
It does; but isn't the status quo also an absolute disaster?
I notice in another reply ITT you stated that you were once employed in engineering medical devices and left that line of work because of the "horrible environment". That's another anecdote I'd use to support my opinion.
I think the status quo is a trade-off between too easy and too restrictive. Too easy kills people; too restrictive kills different people. Saving more of one set of people almost automatically means killing more of the other.
Re the horrible environment: It was horrible in that we didn't (at the time I was there) have much automation to handle the FDA documentation requirements. (Don't think the standard "document your code" here; FDA requirements are much more than that.) The result was a mass of manual paperwork for the software engineers, which (at least for this one) is a horrible environment to work in. It got better after I left, as they got more tools to help them.
> That's another anecdote I'd use to support my opinion.
But I don't see it as supporting your opinion, and I'm the guy it happened to.
Your statement I just quoted above is FUD. I'm not arguing that the OP link describes the best way to reform the FDA, but I refuse to believe without substantial proof that there is no room for improvement the FDA medical device approval process.
>But I don't see it as supporting your opinion, and I'm the guy it happened to.
But it's my opinion. You get to make and have your own opinion, but not mine.
Here's what I took from it: A likely competent software engineer quits field due to onerous work requirements.
A lot of assumptions are being made here, but your anecdote isn't dissimilar from other ones that I have heard, and negative anecdotes are in the majority WRT this field. Overriding conclusion is that better engineers are avoiding or leaving the field; leaving those jobs for less-capable candidates and driving costs up.
I never said that there is no room for improvement. I said that changing things so that devices are released faster almost certainly means that some number of devices get released that are unsafe. Changing things so that devices that are released slower almost certainly means that some other number of devices get held back that would have been safe. I take no strong position on where we are in relation to the optimum spot. My personal opinion is that the FDA was in about the right spot about 6 years ago, but I am not dogmatic about that opinion.
> Here's what I took from it: A likely competent software engineer quits field due to onerous work requirements.
Thanks. I'd like to think that, too. But it's also possible that an engineer whose temperament was to move too fast and not careful enough found that he was in the wrong environment.
The flip side is that FDA approval would grant the drug maker immunity from lawsuits over deleterious consequences of the drug.
Of course, the drug maker would not be allowed to make fraudulent claims in either case.
The much-touted ballpark estimate of a billion dollars to produce a modern pharmaceutical from start to finish is about a decade old now - that figure is adjusted for today's diminished dollar value, eroded by inflation. You can replace "pharmaceutical" with any medical technology that is going to require a fair amount of original research and further tinkering in the laboratory to obtain the first working prototypes and the result is much the same. This large round number is the kitchen sink cost across a decade of work, including failed attempts and the opportunity cost of investment. The lion's share of the direct expenditures are imposed by regulatory requirements: trials, data, and more trials.
It has long been my position that almost all of the work carried out in the US at the behest of the FDA to prove safety is unnecessary. In fact it is counter-productive, as the immense imposed costs on development shut out a great deal of the experimentation and small-scale initiatives needed for rapid progress. Much of what the FDA demands is not demanded by similar regulatory bodies in other parts of the world, and even their requirements are very onerous in comparison to the standards in place fifty years past, a time when medical development seems to me to have worked just fine. It is a question of balances and choice: it is better to err in favor of faster progress and informed patient choice, but that is very far from the present state of affairs.
The result of an excessive and growing regulatory burden is that many potential medical technologies languish, are rejected outright, or are never developed at all, and patients suffer as a result. Our future health is determined by the pace of progress, and a slowdown across the board harms all of us considerably. Unfortunately that cost is invisible to the public at large and thus bureaucrats suffer very little as a result of the harms they cause by blocking progress. Meanwhile even comparatively small harms caused by an approved treatment that turns out to be overwhelmingly beneficial save for some negative effects for some people can snowball in the media to cause great damage to a career in the FDA bureaucracy. So you can see that the incentives are very much aligned with ever greater demands for proof and ever greater costs imposed on medical research and development. This is in fact what has happened over the past few decades, with the present result that in an age of radical progress and plummeting costs in the laboratory it is nonetheless the case that medicine is ever more expensive and the introduction of new applications of medical research has slowed down. This is well known and widely commented on, but so entrenched that this detrimental trend shows no signs of slowing.
So, to return to the cost of developing a modern pharmaceutical product: estimated at a billion dollars (in today's dollars) ten years ago, the same approach results in an estimated $2.5 to $2.8 billion dollars now. Interestingly the bulk of that is apparently not due to the increased time required to run the regulatory gauntlet, but rather largely due to other increases in the demands of regulators: much larger trials and more data.
http://csdd.tufts.edu/news/complete_story/pr_tufts_csdd_2014...
http://lifescivc.com/2014/11/a-billion-here-a-billion-there-...
There is great frustration with this state of affairs, and it manifests with the creation of organizations such as Faster Cures and Tomorrow's Cures Today, none of which seem to be making much difference, sadly. The lobbying makes some people richer, and nothing changes as a result. In the end the only thing that is going to work is regulatory arbitrage - that the development goes to other parts of the world, and therapies are available there until the FDA is shamed into approving them because the people in charge look like clowns if they don't. This is exactly what happened for stem cell therapies; those available in the US today would still be held up in interminable trials and requests for more data without the fact that they have been widely available in overseas clinics for years. The pattern of activity in the FDA has nothing to do with safety and efficacy and everything to do with job security for bureaucrats.
What penalty does the FDA face?
For example, take the first Rotavirus vaccine (Rotashield), which was associated with intussusception (it's no longer clear it even caused it). It was pulled from the market, but Congress definitely got involved, an newer, unrelated rotavirus vaccines have very, very active post-market intussusception surveillance studies associated with them.
http://nationalpainreport.com/activists-want-fda-commissione...
So now, anyone can sell medicine, but if they want to claim it's effective for some disease or condition they have to actually back up the claim.
Some examples:
"Many high-risk medical devices today are approved on the basis of just one clinical trial (as opposed to new medications, which usually require two trials)"
This is only true if the medical device in question is an update to an existing medical device that has extensive clinical data and use in the field. Sometimes that minor update is as minor as a software modification.
"And only a small minority of clinical studies of medical devices are randomized, controlled and blinded — the gold standard for reliable evidence"
This is true because it's unethical to conduct major surgery on someone to give them a placebo, so we can't really blind the studies. They are definitely randomized and controlled though.
"In fact, according to a 2014 journal article co-written by one of us, the F.D.A. has never issued a warning letter or penalty for a postmarket study delay."
This is laughably untrue. I've in fact worked for two companies that received penalties for not being timely in postmarket surveillance. It might be true that FDA has never given a warning letter (which is their version of 'you've been a bad boy, no new products for you until you get your shit together) solely for postmarket reporting failures, but that's a much different claim.
"When a high-quality clinical trial was finally completed, in 2011, it found that patients who had the device implanted were more likely to have another stroke and to die than those just receiving medical management. Despite this evidence, the F.D.A. did not withdraw the device (though it did narrow its recommended uses)."
Wingspan was left on the market because for certain classes of patients it was a good treatment. FDA's action was appropriate.
"But alarmingly, the 21st Century Cures Act would establish a third-party program of nongovernment authorities to assess whether a company is permitted to make such changes. The act would enable the device manufacturer itself to select — and pay — the third party from an approved list. This flagrant conflict of interest would make it impossible for physicians or patients to have trust in the safety or effectiveness of updated medical devices."
This is the mechanism that Europe uses to approve all medical devices, not just changes, and it is not the nightmare scenario the authors make it out to be. The reason is that the regulatory middlemen have balanced incentives: they can't be too lax because when they get audited by the European Commission they can lose their license (and in effect go out of business). They can't be too strict because a manufacturer has 5 other companies they can do business with.