The FDA's notes from its visit to Theranos' labs don't look good
businessinsider.com
businessinsider.com
I'll speculate that there's something so distasteful about Theranos --the wild mismatch between valuation, amount raised, and deliverables-- that these two investors felt that it threatened the ecosystem itself. Like they needed to distance Sand Hill Rd from this stinker before it taints the other (still unproven) unicorns.
Or maybe it was something else, dunno. Anyone else notice it?
Investing in 9 sham companies that fold and 1 unicorn that pays out at 10:1 or 100:1 is more lucrative than investing in 10 companies with modest, but reliable RoI's. Peter Thiel's Facebook investment paid off at 2000:1, Accel's facebook investment paid off at 800:1, Andy Bechtolsheim's google investment paid off at 17000:1, Andreessen Horowitz made 300:1 off of instagram. If you've got a good enough pitch you can get as much money as you want.
This one deals with the requisite post-market aspects (surveillance, complaint remediation, etc).
For a Medical Tech. start-up to fail on post-market surveillance is in-excusable. Move Fast & Break Things should not apply to people's health and medications.
There is something very wrong when a company with that much cash fails BOTH in pre & post market situations.
(My wife has worked in this area for the past ˜15 years, in both clinical & post-marketing environments. It's serious business.)
Are you confusing it with post-marketing commitments to run additional clinical trials? That's not the same thing as post-marketing surveillance.
[1]http://www.fiercepharma.com/story/roche-overlooks-80000-adve...
"Under the current system, the FDA has the authority to revoke approval of some drugs for which Phase IV commitments have not been honored. This authority has never been invoked."
I mean there are 2 Theranos centers smack in the middle of SV (Palo Alto and Redwood City) - the WSJ's exposé centered largely on actually trying the tech and finding others with disparate results - where were the SV / SF / Oak proper journalists on either end of this story actually scrutinizing a $10b monster?
"Watching the media build up and tear down Theranos feels the same with the sign bit flipped. Little data, and lots of hyperbole, both times."
c/o Sam Altman's twitter and he's spot on.
Most journalists are not capable of narrating how e.g. a Rails application works. They're equally not capable of walking an interlocutor through e.g. Phase N clinical trials. They rely on people who actually have domain expertise to tell them what the ground state of the world is like. One would hope that a good journalist goes around and checks those representations, but this is not always true.
Every interaction I have ever had with a journalist attempting to use me as a source has decreased my confidence in the profession. I'm on like six organizations' internal lists as "Speaks English and can answer any Japan question." (My qualifications for this: I speak English, have a professional background which suggests bare minimum competence in a field which was once newsworthy regarding Japan, and then was in the NYT in that capacity. "Close enough!")
They've asked me to provide my opinion of everything from e.g. Abe's economic policies to whether the man on the street thought that the Japanese women's soccer team was counteracting the national malaise after the Tokoku earthquakes. When I give them the honest answer, which is that I'm a small software entrepreneur who does not know anything useful about nation-level economics policies and is sufficiently politically apathetic that me knowing that Abe is prime minister actually means that I must be paying more attention than usual in 2015, that does not result in them stopping the interview or taking me off their list.
Journalism is a high-prestige field which, to re-use Michael Creighton's observation, we understand is virtually never right about even the simplest consequential facts about things we actually understand, but we trust it to be accurate with regards to consequential facts about other fields, and this trust endures through every report that we have about journalists being catastrophically wrong. The alternative to trusting journalism is to believe that the New York Times and BuzzFeed hire the substantially same people to do substantially the same work for substantially the same reasons, and that the NYT is prestigious and BuzzFeed is not because the NYT has been BuzzFeeding for centuries.
There is some threshold of evidence that causes us to accept the BuzzFeed hypothesis, right? Right? God help me, I know I sound crazy, but I think I'm there.
Why would they take you off their list if you gave them an honest answer (I do not know)?
You clearly knew were to stop, so you did not misinform them . That is a valuable trait for an expert.
There is some threshold of evidence that causes us to accept the BuzzFeed hypothesis, right? Right? God help me, I know I sound crazy, but I think I'm there.
Welcome to the ranks of conspiracy theorists. Anyone willing to admit difficult truths eventually winds up here.
Is there a time in the history of journalism where journalists were ever right about even the simplest consequential facts? That would at least explain why they're still considered a high-prestige field. Anyway, I'd love to find out how such a worthless field earned and continues to maintain it's prestige.
[1] http://www.nytimes.com/2015/06/07/magazine/the-agency.html
Tell me, cpach, why did you feel the need to defend the news establishment? You obviously think there's some value they're bringing to the table. What is that value?
But as I said in another comment, how else would you have any idea about what happens around the world? Do you see no value whatsoever in publications such as The New York Times, The Washington Post or The Economist? I'm not talking about The Sun, Fox News or Russia Today.
I see a value in organizations that have the funds to pay for investigative journalism. A recent example is the private jet scandal within the Swedish manufacturing company SCA.[1] The newspaper Svenska Dagbladet ran a series of articles about how the upper management at SCA misused the company's private jets, costing the company loads of money. Unsurprisingly the shareholders were not happy and many key persons had to resign because of this.
Had it not been for the articles in Svenska Dagbladet, how would the shareholders ever find out about this misuse of the company's resources?
[1] http://www.thelocal.se/20150123/one-of-swedens-most-powerful...
Since there's nothing I can do about anything anywhere else in the world, why should I care? Furthermore, if very little the news establishment publishes is correct, how can you say you have any idea what's actually happening around the world? You don't. With the readiness with which the news establishment will publish absolute junk, what makes you think they won't publish what they're told by their corporate owners or the government? When nobody does any fact checking, it leaves them (and you) vulnerable to manipulation.
> Do you see no value whatsoever in publications such as The New York Times, The Washington Post or The Economist?
None.
> Had it not been for the articles in Svenska Dagbladet, how would the shareholders ever find out about this misuse of the company's resources?
Wow, you had to go all the way to Sweden to dig up an example for this? We're talking about the American news establishment. I have no comment on or exposure to European news. In the American media, investigative journalism is a dead art. The disclosures we see in the news are mostly done because we have a (more or less) open market which gives an incentive for all sorts of people and organizations to profit from these sorts of disclosures. That part of the system would work just as well without journalists.
All the way? Has it occurred to you that not every participant in this forum is a US citizen?
It looks like this could be solved very quickly if all parties were interested. I am quite sure that Theranos has made some powerful enemies, many of whom probably have some influence within FDA...
[0] https://www.theranos.com/news/posts/statement-from-theranos-...
But I can see how things like lipid panels would have contamination problems with finger-stick draws, although they say they use a combination of those and "venous blood draws using smaller needles and smaller tubes", which doesn't strike me as a big selling point. In my experience, pain is associated with not hitting the vein right the first time, and once you hit the vein, you might as well draw more than enough for the planned tests and any followup tests ordered once the first results come in.
That's new, they used to claim they could do it all from their Nanotainer sample. It was and still is a crock, but that's entirely what Theranos' claim to fame is.
Our tests require less blood and less pain, regardless of whether we use our proprietary finger-stick technology or a venous draw.
We do venous blood draws using smaller needles and smaller tubes.
Now, I will say that when Quest does a venous draw they use a honking big tube ... well, the normal size since they stopped using syringes (you young whippersnappers are lucky, let me tell you!), and draw well more than a mL, but for the latter, who cares? Especially if that allows them to do followup tests without requiring another draw, as happened with me a couple of months ago. And for that matter, the larger the needle, the shorter the time is has to be in you to get a given sized sample.
$10 billion valuation, and this is the best they can do?
It seems like silly season is going to end soon. This time there are a lot of solid businesses in the valley with solid revenue and solving real problems. Then there are the dilettantes and the scammers. All will be revealed as it becomes more difficult to raise money.
Do both have the same guy working on FDA-approval?
this one is the most damning. FDA gave them the opportunity to correct this, and they sat on it for a year.
The culture of move fast and break things is fine for consumer products, but the cavalier attitude with people's lives is overstepping.
I worked for a medical device manufacturer for about 15 years. 21 CFR 11, and the test & quality tracking requirements, make it extremely hard to pivot quickly.
There is an air on incompetence about this whole thing, and Holmes isn't doing herself any favors battling with poor evidence against WSJ in public.
However it is entirely possible to be agile and (relatively) fast in this space - not by breaking rules but by designing the process to be light where it can, detailed where it must. And then paying attention to how it is working and improving it constantly. You can probably cut time to market in half relative to industry averages this way, which is nothing to sneeze at.
I count this as different than the 23AndMe situation where they're arguing with the FDA over how the results can interpreted and passed on to the consumer, that's a difference in opinion that doesn't rest on the quality of the testing science.
that's a difference in opinion
Indications for use aren't a matter of opinion. 23AndMe clearly screwed up there, but I agree it was a different nature of screw up.Well if they want to hire folks out there there's a huge amount of capable people ready to be poached from other companies.
I would theorize there's no shortage of biochemists or lab techs. The graphs seem to show a gradual increase in unemployment and a gradual decrease in employment for life sciences PHD grads. 2010 was a crossover year where afterwards less than 1 in 5 grads will get a job but more than 1 in 5 will be unemployed. This doesn't count underemployment; a biochem PHD working as a waitress is technically employed.
If only the upper 20th percentile of the grads currently get jobs, lowering one companies standards to accept the 21st percentile of grads is still pretty far along the right side of the bell curve... they'll do fine. Regardless of the details, its hard to describe a market where 80% don't have industry jobs as being limited by the labor pool.
E.g.
> Observation 1: "Design validation did not ensure the device conforms to defined user needs and intended uses."
You don't build a bridge without defining the maximum carrying load you intend to use it for, or validating that it will carry that load. Getting lucky and having the bridge not fall down (at first) doesn't ameliorate that process defect.
P.S. I haven't been following the Theranos story closely. I am taking the fine article at face value.
> The government doesn't give you "feedback."
http://www.fda.gov/ICECI/Inspections/ucm256377.htm > when an investigator(s) has observed any conditions
> that in their judgement may constitute violations
Sure sounds like feedback to me. > The process documentation and design specification
> and validation issues are not optional
No, but they can be iterated on. > I am taking the fine article at face value
The "fine article" presents small snippets of a heavily redacted article. There's not that much value to take...What's interesting about Theranos is the hints that the technology may not be ready for prime time yet either. In which case moving more slowly, and taking that into account in the funding stages, is not just a reality of compliance but a reality of engineering. This stuff is hard, MVPs aren't enough, and successful ventures are going to take longer to yield a salable product than in other fields.
It's junk science, has been since day zero. Her father's USAID connections make me think there's more going on here, and that they failing could embarass elements in government.
My understanding was that this company administers innovative ultra-small volume blood testing. Now they say that only works for the HSV-1 test (that nobody wants, and by some accounts already existed), and for the rest of the 240 tests they use normal lab procedure.
What is the great innovation here? What is it that this company has been doing for 10 years? Why is it worth $10B?
A fully tested process? Surely you jest. Hell, you don't even need to have an idea to get funded[1].
It was enough for her to have some innovation that could plausibly work.
1. https://en.wikipedia.org/wiki/HP_Autonomy
2. http://www.marketwatch.com/story/the-mysterious-case-of-hewl...
but at the end, they did a decent job specializing in electronic records management & e-discovery / litigation holds. Not nearly enough to justify the ludicrous price, but at the time HP was desperate for something to help them compete in any sort of "big data" market.
1. http://www.theonion.com/video/hp-offers-that-cloud-thing-eve...
2) What does the $10B valuation people keep referring to has to do with anything? If they do not meet requirements should they be valued less? This does not make sense. If you do not meet standards for xyz, you shouldn't be allowed to perfom xyz. The feedback should not be around the valuation.
3) The same press that are knocking Theranos are the same press that were calling it disruptive. If anyone is to be blamed it should be them, and the government agencies that allowed testing.
We know that for all the test except one, Theranos uses usual diagnostic devices (from Siemens etc.) like any other diagnostic lab. So it will be something like ~99.9% of their tests are done on traditional devices. So we can safely assume that those tests are as reliable as those from other labs.
So if they are same as any other legacy industry for 99.9% of the time, then why such a big valuation?
They should be valued same as their nearest rival (which I’m sure is a giant compared to Theranos), and if adjusted for the fact that whether Theranos is going to upset them without any secret sauce (and with fewer resources) then the valuation should be near to $0.
That the answer could be "not very well" (and we haven't heard otherwise) is what's most troubling to me.
It's say it's worse. I followed up on the general impression they were proposing to do this all with fingersticks, e.g. this citation from Wikipedia's introductory paragraph published in 2013: http://www.wsj.com/news/articles/SB1000142412788732412300457...
Based on my general knowledge of biology and medicine, I'd expect there would be a whole bunch of things you can't test that way due to general contamination from all the normal cells and their membranes that get punctured in a finger stick. Lipid panels, for example. In fact, the large venous draws normally taken, at least for the first tube, could be in part to swamp the contamination that comes from getting the needle into the vein in the first place.
Per other discussions, they're now using their own technology only for HSV-1 tests, which are PCR based and therefore require only a tiny sample, and the above sort of contamination would not be an issue.
Theranos uses non-venous blood for their proprietary tests (currently only 1 test) and rest 99.9% (figuratively speaking) tests uses venous blood in traditional diagnostic equipment.
Back to the same question - If almost all of their tests are done using traditional equipments, then why $9B valuation?
Just because standard practice seems unnecessary to you doesn't mean that it doesn't exist for good reasons. To anyone in biotech or familiar with what can go wrong with physical products, your complaints come across as ignorant.
Document numbers and release dates go toward ensuring everyone at the company is working with the same information. Most computer science students learn about the failure of the Mars Climate Orbiter [1] because one team was using feet and the other meters. Guarding against issues like those is exactly what FDA is trying to do.
Having been involved in several pharmaceutical studies, I can assure you the level of documentation detail demanded is extremely picky. Medical laboratory rules are similarly strict. They have to be given the risks to patients if not closely regulated.
The essence of the requirements are thoroughness and consistency. Medical laboratory measurement devices have to be carefully calibrated in order to be predictable, a lot rides on the accuracy of results. Without conforming documentation of calibration and testing procedures, it's impossible to trust lab test results, which is untenable.
It's not "just red tape", if true that compliance with standards is arduous. I have no trouble understanding reasons people might avoid getting involved in biotech enterprises, or recommending it for others.
Good. We're in mutual agreement. Because I or my friends don't want people working on matters of life and death who don't care about "red tape" or "document numbers".
As far as the biotech industry, the funding situation is awful, especially for early companies, and that is a fine reason to avoid this area. Needing to "say what you do and do what you say" doesn't strike me as that much of an obstacle in comparison.
when you have your shit together in the most basic ways, you simply won't have this level of problem.
we used to call people who didn't do this 'rogue traders' and the like. they were looked down on like scum instead of looked up to as role models.
I'm not saying conspiratorial that this is all BS but rather that I'm surprised there isn't some more skepticism considering disruption of industries this big and regulated will never go easily.
I understand that folks believe the cult of disruption has blinded everyone in this case and think that may well be true. still these issues highlighted by the FDA sounds to me like any scrappy startup which has to cut some corners. some of you will claim the sky is falling but I don't buy it. if there's nothing to their tech then they will indeed collapse but these papers are not enough to make that clear
And this: http://www.cbsnews.com/news/fda-corruption-letter-authentica...
And this: http://seattleorganicrestaurants.com/vegan-whole-food/instit...
And this: http://ceramics.org/ceramic-tech-today/biomaterials/gsk-drop...
And this: https://reason.com/blog/2013/11/25/fda-shuts-down-23andme-ou...
Why anyone's first inclination would be to trust the FDA is beyond me. In fact, this whole story has a fishy Denmark smell to it. Why for instance is all dissenting opinion being down voted?
Maybe if you would have linked the official statements of both organizations rather than a "reason.com" blog link with opinionated bullshitting, then your point might stand some scrutiny.
This and the fact that things have now shifted [1] because even if that doesn't fill your sensational hunger, no, the FDA is not on a crazy trip to stop personal genomics.
[1] http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/uc...
Oh, do you? How about you explain it then.