Theranos Ran Tests Despite Quality Problems
wsj.com
wsj.com
"That letter cited “deficient practices” in five categories. In one of them, hematology, the problems found by inspectors posed “immediate jeopardy to patient health and safety,” regulators said.
The test that inspectors found Theranos kept doing despite the erratic quality-control results was a hematology test that measures how long it takes blood to clot, the people familiar with the matter said.
The test is an important part of the treatment regimen for patients who are at risk of strokes or have blood disorders...
People who have seen the Theranos inspection report said it seems to indicate the company ignored quality-control results for the blood-clotting test that fell short of its own criteria.
The results produced in the quality-control checks repeatedly deviated from the lab’s typical result by more than two standard deviations, those people said...
At Theranos, there were seven quality-control failures in a single day, said a person familiar with the report...
Dr. Hamill said any doctors who received prothrombin time test results from the Theranos lab during the six-month period shouldn’t rely on them and should have their patients retested as soon as possible. “Those results are not worth anything,” he said...
The federal inspectors also concluded that the Theranos lab in California used expired reagents, or substances added to blood samples to elicit chemical reactions, for some prothrombin time tests, according to the people familiar with the inspection report.
Those people said the report also found that the lab’s director didn’t have responsibility for the lab’s quality-control program and the lab’s quality-control manager was unqualified..."
EDITED TO ADD: Actually, the really key sentence is this one: "Theranos spokeswoman Brooke Buchanan said: “The PT/INR issues identified by CMS related to tests run on conventional equipment using venipuncture samples.” A CMS spokeswoman declined to comment." Because that means that these quality control problems were not due to Theranos' "revolutionary" finger-prick tests, they were from regular old venipuncture tests running on a standard machine that were giving wildly incorrect results, expired reagents, no QC in the lab, etc. Bad data is worse than no data. If they couldn't get old-fashioned labs right, no wonder they were having trouble with the finger-prick stuff.
A patient may be subjected to life altering medication or treatments, which is not a trivial responsibility that can be shrugged off with erratic processes or lack thereof.
A lab director's responsibility includes ownership of quality processes, reviewing abnormal results after the fact to see if proper processes were followed (such as communicating with the respective provider), reviewing financial success, and much more. To shrug off quality control and operate as if reputable is not just unethical, but incompetent. Such operations should cease immediately.
Disclaimer: I have experience in the medical lab industry.
[1] http://www.nytimes.com/1983/10/22/us/3-ex-officials-of-major... [2] http://pubs.acs.org/subscribe/archive/tcaw/10/i11/html/11reg...
Will anyone from a 10B startup go to jail ? Probably not, but we can schedule that for afternoon tea outrage.
But they do initiate investigations, and it is a crime to defraud or attempt to defraud the FDA. And if the FDA so chooses, they can forward their information to Justice and participate in the proceedings against the accused. They could provide the evidence of wrongdoing and increase the political pressure to act. They can do a lot to ensure that the state determines it has sufficient reason.
And Holmes and the board believed in good faith that their prothrombin test was accurate. No mens rea, no culpability. No prison for anyone.
I doubt this will result in jail time for the principles, but they should be expecting lawsuits over this.
And, there should be some civil remedy that costs Theranos big-time. Massive fines, one-site FDA overseers, etc.
That said, I don't think I'll be sending my bloodwork to Theranos any time soon. That's some scary stuff.
With that background, your faith in the system is touching but unwarranted.
Think people dying when a building in disrepair burns.
You don't have to mean for something bad to happen to get jail time. You're still responsible for your actions, intentional or not.
I'm not lawyer, so bear with me, but my lay understanding is that if it can be shown that they should have been aware of the quality and other issues regarding their results, they could be criminally negligent. Feel free to correct me.
Generally the FDA will give companies several outs to fix problems, they only bring out the handcuffs after several ratcheting steps (one of which includes bringing in a US Marshal and a team from the FDA, at your expense, to be at your business every day and tell you how to run your company).
These are rich powerful people, they don't go to jail.
It's starting to look very ugly for what started out as something overreaching, and I think she will fight even harder to keep the house of cards from falling but pretty much this unicorn will be the first to go in 2016.
In terms of practicality/Insurance - this would mostly be a function of how absolutely certain we are that the tests are accurate, and how damaging incorrect results would be. If we are 99.99% certain the values are correct, and incorrectly results wouldn't result in irreparable harm - then yes, not worth the investment in redundant lab tests.
But, if there is, say, a 5% chance I might die on an incorrect test result that has a 1% chance of happening - and that test is performed 12 times/year - then you can pretty quickly calculate the net-present-value of my life versus cost of test. (I.E. If I think my Life is worth $10mm, and the test normally costs $500 each time, or $6000/year, and I need to have this test done for the next 10 years, (assuming the value of my life doesn't drop over time, to make this simpler) - then each time I get the test there is a 1% chance of failure * 5% chance of death = 0.05% chance of dying as a result of each test.
The odds of me dying then, in the next 120 test = 100%-(chance-of-not-dying-120-times-in-a-row) = 1-.9995 ^ 120) = 5.825% chance of dying in this scenario using tests that cost $60,000. If I could reduce that to 1% (4.825% reduction) chance by spending an extra $120,000 (perform the test in two extra labs) - then my my investment of $120k would return 4.825%*10mm = $482,500.
You can work the numbers, assumptions, etc.. however appropriate - but for some combination of test costs, frequency, potential harm, and value assigned to life - there is quantifiable value that justifies performing the test multiple times (and, likewise, clearly circumstances in which, even if the test results are more likely to be inaccurate, not worth performing thee test).
Not having access to multiple labs is a real problem though.
In real medical practice if a physician sees a lab result that is way out of range, or significantly different from the patient's previous result, or inconsistent with other aspects of the patient's condition, then she will order a repeat of the same test in order to confirm the results.
Other high-stakes scenarios in which the same tests are run multiple times are for high-economic cost events such as drug abuse, either in the workplace, or in sports.
You even note in your response, that, In real medical practice if a physician sees a lab result that is way out of range, or significantly different from the patient's previous result, or inconsistent with other aspects of the patient's condition, then she will order a repeat of the same test in order to confirm the results. - which is exactly what I'm suggesting people have done - only with a (presumably acceptable) time delay between when the first, and later tests are required. If the results of such tests were that clinical activity had to take place quickly, then you might not have the luxury of a delay, and in which case, multiple independent test vectors would be very useful - if they were economically feasible.
What people sometimes fail to consider - if the test was worth doing once - at, say, $600 - then why wouldn't it be worth doing (at presumably much higher confidence), three times at $100? And, if people agree, "OK, for $600 and $100 that makes sense" - they have to understand, those two numbers are completely arbitrary, and can only be understood in relationship to the value of the results. Whose to say there aren't a ton of tests today that are "$100" - that couldn't be done multiple times to be certain.
Having a slightly more accurate result is useless, because there is far more biological variation in a person (from disease, the vagaries of physiology, compliance, pharmacokinetics, diet, the time of day, mental state, the season etc etc) than technical variation in a lab test. You would be optimising the wrong thing.
The only example I can think of where someone would probably be harmed by this is if they were undergoing some kind of a procedure with a bleeding risk, and the Theranos test said their clotting was fine when it wasn't, and they had a significant haemorrhage during the procedure. Someone will probably launch a class action lawsuit anyway.
I think that the Theranos board will realise that they need to get rid of Holmes in the next 6 - 12 months. Even if the tech actually works, having her there has become a liability.
It's really funny how people have stopped hating on the FDA when Theranos is mentioned. In any other context the FDA is still worse than Satan and the pox.
The instruments can be trusted.
If a test is borderline, either the lab tech will manually retest it, or some instruments can be given business rules like "if result is on the threshold of good/bad, automatically rerun the test and provide both results to the physician."
We were well aware that the software we wrote affected real people's lives and acted accordingly. These idiots are only giving the rest of us a bad name.
All of the critiques I see don't seem even close to show stoppers. No public peer reviewed studies? Some lab updates to match FDA regulations? Using some "unauthorized" third party equipment? This is a new company, why should these be the doomsday callings?
I have no idea if Theranos will succeed, but I can't shake the feeling that this media response is inflated due to the high valuation, and journalists are falling victim to the hype as much as the public, whereas this is a fairly normal health startup otherwise. It probably was a mistake to accept such a high valuation, so I don't think Theranos is totally innocent here.
It's also strange that a paywall article made it to the front page? I wouldn't guess there's a big overlap of WSJ subscribers and HN readers (lol conspiracy).
If we as software engineers want to work on problems that matter, and not just dating apps and ad networks, we can't have a work ethic that encourages us to ignore troublesome laws and regulations and launch half-baked MVPs with a plan to maybe fix problems as they arise in future iterations. Sure, work smart, automate all the boring shit, but your automation has to be correct. Sure, focus on a MVP to get off the ground -- but make sure your product is viable, and correct across the minimum set of features.
Even when we are working quickly, sparsely, we should be laying good foundations for the future. We shouldn't be lazy conmen selling poisonous snake-oil.
All the way. It's not there. At all.
Nobody really gives a fuck about the valuation except valuation wonks. But a lot of people care a lot about this woman being a complete charlatan who is endangering public health.
Absolutely not. Start-ups are supposed to fail early and fail often but when you're in the medical space you simply can't act entirely like a technology start-up. People's lives depend on many of these tests; you can't just go "well let's ignore the FDA and we'll take the slap on the wrist" like in other industries. Here they might deliver inaccurate test results that are then used to treat someone for a medical issue.
That can be deadly. Yeah many people are going crazy from the valuation and secrecy but there is plenty of legitimate scrutiny here especially if you read this article (I'm assuming you didn't because of your very generic Theranos comment versus addressing the issues in the article itself which sound grossly negligent, IMHO).
But the attention is because people can't stand high valuations?
There is a reason why certain industries are scrutinized so closely because of the effect they can have on people. Being in those industries, even as a startup, means you must follow the rules.
Or are you fine with your doctor using some "unauthorized" random equipment the next time you need critical tests or surgery?
Hm. In that case I guess we would have to ask the question -- how legitimate does the eBay site look?
Theranos is NOT a new company. They've been around for more than a decade.
This isn't the latest and greatest dating app. This is medical tests that have a very real impact on the treatment provided to patients.