U.S. Probes Theranos Complaints | Hacker News Reader

U.S. Probes Theranos Complaints

wsj.com

102 pointsjerryhuang10042 comments

What's notable about the meat of the complaint sent to the FDA is that it's less "this technology actually doesn't work" and more "this company has no culture of experimental integrity".

From the article:

"hadn’t fully assembled the proprietary machines used for the herpes study when the experiments began" ... "company underreported the rate at which the machines broke down during the study" ... "employee also alleged that some crucial parts of the devices ... were modified to improve their accuracy"

FDA is generally skeptical of validation data not generated from "production equivalent" technology, but they get even more skeptical if you tell them something is X and it turns out to be Y down the road. That (rightly) sends them on a fishing expedition.

You only fudge the results if the technology doesn't work. The FDA does not look kindly on such shenanigans.

Not necessarily to the former. Definitely to the latter.

To elaborate point 1:

- If the crucial parts of the devices that "were modified to improve accuracy" were modified early on in the study the data could be representative of the final product.

- The same applies for the devices not being fully assembled. If they weren't fully assembled, and the data were still good enough to defend a 510k, then the data should represent worst case usage, unless full assembly would somehow make the technology less performant.

Yes it is always possible their one test works, but reading between the lines it appears that Edison is flaky and hard to get to work consistently. Things like changing the pipet tips halfway through are amateur hour - issues like this should have been picked up in R not D.

really sounds if they went to certification's stage too early, regulatory bodies do not tend to support rapid iteration of products

The company is already 12 years old.

I fully understand that part, perhaps the FDA needs to adapt as well. Testing doesn't need to be static, there is room for cooperative certification and corrective actions to insure the test completes. With the rapid advances in some areas of technology you could run into a situation where you obsolete the technology you are working on as certification issues reveal improvements.

While the medical field certainly has additional concerns I do see a time where, with proper processes in place, that the final delivery may not exactly be what entered the certification phase.

But they seem to have pivoted several times already. The founder's original patent was for a wearable device that transmits readings to your doctor.

> regulatory bodies do not tend to support rapid iteration of products

I disagree. 21 CFR 820 is deliberately written such that it does not prescribe any one specific development process; AAMI TIR-45 for example aligns agile methodologies with medical device software development and is an FDA recognized consensus standard.

What regulatory bodies do demand is that one document what one will do, and then, actually do what one has documented. Sounds like Theranos had trouble with that.

Thank you for pointing this out. You'd be surprised how many people even within medical device think design control necessarily means some slow-as-hell waterfall development method.

>Edison machines would sometimes produce “radically different results” for the same patients, the former employee alleged. Referring to a thyroid test known as thyroid-stimulating hormone, the employee wrote that “a patient would swing between” hypothyroidism and hyperthyroidism, or too little of the hormone and too much, when the test was repeated the same day.

This sounds really bad. When you have this huge amount of uncertainty in results, it calls into question their whole finger prick blood sampling methodology. I'm starting to think that it's impossible to get an accurate blood work from a finger prick only.

It is. As someone who's been through the pain of this exact lab fuck up (though from traditional, not theranos equipment I would assume) being mis-diagnosed and starting medication for this condition incorrectly can be terrible. The cure for Hyperthyroidism is so easy, and the condition so common, that people forget what the effects of the extremes of the conditions are.

There are lots of industries you can disrupt, or fail fast at, but lab testing isn't one of them.

What is the cure for hyperthyroidism? I'm wondering if you mean hypothyroidism?

If my cat is anything to go by, the options are: a daily pill, radioactive iodine, surgery or a new type of very expensive cat food.

I think it's safe to say the last one isn't for humans.

Depends on the clinical situation.

The options include partial thyroidectomy/ radioiodine therapy / carbimazole, propylthiouracil and a few other niche options for use use in special situations like pregnancy associated thyrotoxicosis.

>I'm starting to think that it's impossible to get an accurate blood work from a finger prick only.

It is possible, but just for certain test types and error rates. The basic problem is the blood drawn from a finger prick is affected by exactly where it goes in and how much skin tissue is picked up. This variability gets averaged out with a large venous draw and so is not quite as bad - there is still a large variability with even normal blood tests for certain test types.

The tests that work OK are things where you just want a yes/no answer (do you have HSV1) and very poorly for things where you are wanting to measuring exact levels of something.

None of this is new to the industry.

I wonder if they can detect the level of known impurities and correct for that.

If the issue is skin contamination, measure the level of some substance that is correlated with that contamination, then apply a correction factor

I assume that is what they hoped to do. Lots of people with decades of experience in this area have tried to do this before without too much success. This doesn't mean it is impossible, but if you are claiming to be able do something that everybody else has tried and failed then your data needs to be impeccable. No hiding behind trade secrets and saying trust us.

If I remember that anecdote correctly, Edison was producing very variable results on diluted venous blood not finger prick.

It would be good if there was more clarity on whether the test was repeated on the same specimen or on another specimen collected from the same patient at a different time.

If it is the latter, case marked variations could have a number of explanations, ranging from the inevitable inaccuracy of finger prick collection, through errors due to dilution by Theranos, problems with the technology and equipment and diurnal variation of TSH levels

See also Diurnal variation in TSH and free thyroid hormones in patients on thyroxine replacement. http://www.ncbi.nlm.nih.gov/pubmed/2588938

Compared to the other theranos article on top of HN right now, this one is far more damning. The whistle blower is claiming that even the herpes test, the only test the FDA has approved, had its results altered in some way to pass the FDA's requirements. If this is true, what does theranos even have left?

But I was told herpes is forever?

To bypass the paywall: https://www.google.com/search?q="http%3A%2F%2Fwww.wsj.com%2F...

Didn't help.

>To Read the Full Story, Subscribe or Sign In

Strange. Worked for me after I was blocked, same browser. I wonder if you're using Tor, or if they're geotargeting the way they handle google referrers. Or perhaps you're coming from a google.co.XX domain instead of google.com?

Worked fine on my end.

change the https into http.

This is what the "web" link at the top of every story is for.

Just learned something new.

The reason for taking larger samples on many tests using "normal" methods is not that the instruments are not sensitive but that one needs to draw enough blood to get a representative sample of what's actually inside your body. A little prick of blood that comes out is often not such a representative sample. I always wondered how Theranos was going to get around this fact of life. Now as things unfold it increasingly sounds like they can't... they just wanted us to think they could.

I predict Theranos will be out of business within 18 months. It's a vaporware company with a cult of personality founder. Hard to pull off with increased media exposure, even harder to pull off with regulatory agencies looking into you.

Update: too bad Theranos isn't publicly traded, I'd love to be able and bet against them.

So we have a 12 year old company here, developing a machine to test blood (which is altered for testing), has only received approval for one test, which is now being doubted again, has repeatedly lied about their technology and capability.

I don't give one sordid fuck about Holmes, let's talk about Theranos. How is this a successful business in ANY capacity? Has it made a DIME that didn't come from an investor? Does it have ANY reputation that doesn't involve a spunky CEO?

Has Theranos broken the law?

Not a lawyer but "the study submitted by Theranos last year to win the agency’s approval for a herpes test was tainted by breaches in research protocol" sounds iffy.

Falsifying data to obtain regulatory approval is definitely illegal, and federal regulations allow the government to hold corporate officers personally liable.

It has yet to be proven that Theranos did in fact falsify data, though.

Given how things are looking right now, Elizabeth might want to start preparing a shipment of orange turtlenecks to be delivered to her in the big house.

She is very well connected. I will not be surprised if she walks away from the mess in black and not orange.

So conflicted about all of the rage and drama around Theranos. On the one hand, it smacks of the conventional backlash against solutions that revolutionize the way things have been done forever, and there needs to be space for innovators to work through kinks to really bring an idea to market. On the other hand, it sounds like this product does not work (or does not yet work well) and it's not the kind of solution space where errors can be brushed off lightly.

The complaint was a former employee. That gives me some reservations as about the motive.

Thought I'd put this in here: https://www.youtube.com/watch?v=rGfaJZAdfNE

This is EH's response to the WSJ article. Very positive defence of EH. Just wanted to add this for an alternative take.

Well, let's see how she responds to this Federal investigation that essentially accuses her of being a fraud. If her company really did fudge FDA testing results then that's beyond bad.

Reply on news.ycombinator.com