Scientists skeptical about Theranos blood test
businessinsider.com
businessinsider.com
What skills did she have that allowed her to do this, and where did she get the money to hire all the needed PhDs and get through FDA testing? How would she have even gotten access to the equipment and test subjects to prove that her idea was worth investing in outside of an academic setting? The whole thing sounds crazy.
Edit:
I read Elizabeth Holmes Wikipedia page and it says that when she was a sophomore at Stanford, her Chemical Engineering professor encouraged her to found the company. But I can't find anything about about any breakthroughs she made that warrant the millions in early investments. No one seems to want to talk about what exactly it is they're doing, and they're board of directors has a former Secretary of State, a former Secretary of Defense, and a former Senator. The fact that she's also been able to maintain 51% of this company is also astounding.
It takes more than intelligence to start a Biotech company. You need money, equipment, skill, and usually experience.
Medical breakthroughs usually require years of research and skills that are acquired working on a PhD. I'm not saying it's not possible for a 19 year old to make a world changing discovery, but when no one will say what exactly that world changing discovery is, I get a bit skeptical.
Classic disruptive startup, very impressive if you dig down a bit further.
I don't think you can call this a classic disruptive startup. Look at the board members; they are some of those most powerful people on the planet, and yet the founder still maintains 51%.
Either she is insanely charismatic or she discovered something earth shattering (either a novel discovery, or a novel use of existing discoveries).
If you've got the money, you too can have a board like that.
"Give me $16 million. I've got an idea for a new way to do blood tests. It doesn't work yet, and I've got less than a year's experience in the field. But I'm wicked smart because I learned Mandarin as a child, and my professor believes in me. Oh and by the way I'm going to maintain control of my company and never want to give up more than 50%"
That surely didn't happen. She must have something that at least looks very impressive to have negotiated those terms, but what I don't understand is why no one is talking about what that is. They have patents to protect them. Why aren't they releasing peer reviewed studies?
Also, those investors have likely done their due diligence and liked what they saw.
From further reading, her initial idea was just that, an idea. She thought it would be possible to make a handheld device that could do blood tests.
How you go from that to $16 million without the slightest demonstrated ability to execute is my question.
Reminds me of: https://www.youtube.com/watch?v=DkGMY63FF3Q
There's a very good reason to be skeptical when the most anyone can say about the company is the composition of it's board - noteworthy, but I wonder how often that was used to pitch to each of its members.
It's worth rereading the story of Enron when parsing things like this - there's a lot of parallels.
I would agree that you usually do not raise that kind of money without reasonable DD, but there are more than enough counter examples out there of people bullshitting investors out of very large sums, both deliberately and through deluding themselves.
Or just kept executing well.
The company is 12 years old. The initial investments and staffing were likely modest and provisional.
But going back to investors year after year, always having delivered what you predicted at each step, and proving that you've accumulated proprietary technical advantages, will get you to a giant company.
The initial investment was $16 million from what I've read. That's a lot of money to give to then 20 year old with no record whatsoever.
Per that account, by the time Theranos raised its first $6 million in late 2004, Holmes had been working on it for over a year, had at least one patent in process, and had convinced her chemical engineering professor, a dean of the Stanford engineering school, to be an advisor and board member. (He's since joined Theranos full-time.)
So it's not, "kid with nothing but idea and moxie handed money" – but rather "star researcher impresses and recruits a bunch of domain-experts, which then attracts investments".
When she got that first $6 million, and another $10 shortly after, she had no product, no experience, and it wasn't even clear if the technology to do what she proposed was feasible.
I've read her first patent, it's super broad and covers a huge number of possible implementations. There was no way for her to know at the time if any of it was really feasible. And none of it was feasible at the time--it just wasn't ready yet as evidenced by the fact that it took 10 years before anyone heard anything about it.
It took 11 more years of research to get to anything remotely commercially viable. She basically convinced people to pay her to do 11 years of pure R&D. Name me one other founder who has been able to do this with no academic credentials, and no record of success--much less a founder who was 20 at the time.
>"star researcher impresses and recruits a bunch of domain-experts, which then attracts investments"
She wasn't a star researcher at the time. She had almost no formal training, no publications, and a year of experience as an undergraduate.
Edit: I'm not the only one who is very skeptical.
http://pathologyblawg.com/pathology-news/new-yorker-theranos...
Had they waited for the decade-later opinion of a layperson web-developer who's skimmed the first patent abstract, they might have missed out.
What the company does now has almost nothing to do with her original patent applications, so it looks like her original proprietary research didn't amount to much.
They paid a smart person millions of dollars to essentially build a research lab. And after 11 years of research, they now have a way to maybe do some blood tests with a finger prick instead of a blood draw. But there are still questions remaining about the accuracy of these very small sample sizes. Some of the doctors and lab techs opinions I've read, say that even if the technology is flawless small finger drawn samples are much more prone to contamination. The article mentioned on HN even mentions that their tests would be more sensitive to inaccuracies caused by poor operator technique.
How many PhDs out there could have done something similar with 11 years and $400 million.
There are also plenty of experts who are skeptical of Theranos' claims. http://pathologyblawg.com/pathology-news/new-yorker-theranos...
Also you can read here about a first hand account of their procedures. It doesn't sound nearly as revolutionary as the hype is making it out to be. http://decibio.com/blog/theranos-small-sample-big-opportunit...
I'm not affiliated with Theranos, but I see that they are hiring for a large number of open positions, including some related to your areas of professional expertise: https://www.theranos.com/careers/software-development
Every time I read about these ultra successful people, I just think they are freaking machines... and I don't mean that in a derogatory or mean way.
I am genuinely amazed people can operate like that. I know I can't.
>When she was 9, her family moved to Houston, where her father had taken up a job with Tenneco (Oil Company). Intrigued by their father's work in China, Elizabeth and her brother learned Mandarin Chinese at a young age.[5] She spent her teenage years in China, and while still in school, started a business selling C++ compilers to Chinese universities.[8][5]
If their concern is genuine (rather than a way to get a look under the hood of a competitor for the current sellers of such services) there are multiple ways in which they could get their verification without inside information on how Theranos does its thing.
I'm not sure how available the tests are yet, but if it's available really anyone could test it. However, it requires a sample of a large diverse population to get statistical significance and that isn't cheap, which is why most academics probably aren't interested in doing it.
The company being non-cooperative is it's right - it's just not good science.
When the results come back, tabulate and compare.
If there are major discrepancies, they'd probably turn up in the first few dozen tests. But you could run this for hundreds of people with a single analyst, and just a few doctors soliciting their patients' participation, and without needing cooperation from Theranos.
To be valid, you'd want to go in and supply a bunch of baked-samples which should produce a certain definitive result, without the tester knowing this in advance.
Storing 'baked-sample' blood, or doing extra tests against people who weren't otherwise needing tests, couldn't possibly be as representative. Diverting a few drops from tests that are already going through the traditional process, for research/calibration purposes, is likely already authorized by existing patient relationiships. Your "n" for analysis, across any disorder or rare test, would quickly exceed what's "recruitable" for some formal study.
Look at the list of Theranos partners/customers who have brought them to "cash-flow positive": hospitals, drug companies, insurance companies, and the US military.
It's naive to the ways of industry R&D to assume that such constant cross-validation hasn't been (and isn't still) happening.
Proving that it works is a much harder problem that probably only Theranos is willing to finance.
Proving reliability does rely on examining internal methods, and showing they are consistent and reliable.
I concur it somewhat lends to Theranos' credibility that no competitor is disproving their accuracy.
So the right answer is to repeat the experiment. Perhaps many times. You'll end up with a statistical judgement of consistent inaccuracy and while it'll be easier to prove (in terms of statistical strength) than showing accuracy, it'll still be (a) reasonably expensive, (b) generalizable research, and (c) less punchy for headlines.
(b) is especially concerning because if you publish generalizable research involving humans, as would be the only way to do this, then you have to treat it like a legitimate clinical trial and are responsible for many more legal processes, e.g. IRB approval.
I'm assuming they've done this study, but have refused to publish it for some reason. But this is the real question from the article - where is this study? You don't have to disclose the full technique for the test in order to publish the comparison.
> I'm assuming they've done this study, but have refused to publish it for some reason.
Because they're a bog-standard vertical integration play and if that fact comes to light before they are done using the hype train to raise money and grab market share then their competitive advantage will evaporate overnight. If they only release test results to interested parties under NDA it prevents public sector science from rebutting their claims to disruptive rather than incremental innovation, allowing the hype train to steam ahead unimpeded.
(note that I'm not saying that Theranos hasn't conducted studies, rather the FDA doesn't require any)
I have followed her career. In every Interview, she talks about the huge about of funding the company has received.
Why not perform double blind studies? Blood is sent to Quest Labs. Blood is sent to her companies machines. The results are collated by a third party. It seems like one of the easier studies to set up? It seems like a cheap way of proving her machine works as described? Plus, the company Lawyers could sleep at night?
That should be a little bit of a red flag to anyone - since the history of scientific fraud is full of lone specialists who couldn't brook explaining the technical details to anyone else qualified to understand them.
Now, I'm not sure if that's because:
A.) They don't have a game changer or it's going to come under fierce scrutiny by the larger scientific community so they need to try to mandate its use through legal means and using the military as a gateway and OUR military budget as an early source of funds.
OR
B.) Even though they have the real deal, the nature of the political landscape in which medicine operates necessitates the back-room dealing that this board can deliver on.
But that's one thing a successful theranos could do, and foreign policy is important at that.
You can see here that they have replicated most of the routine blood tests out there. Interestingly, they have also replicated some of the archaic and pretty useless blood tests like Rheumatoid Factor. They also offer Erythrocyte Sedimentation Rate (ESR). This is pertinent because ESR is actually a physical property of blood, it is performed by measuring the rate at which red blood cells fall to the bottom of a tube. Perhaps they have developed a microfluidic method to measure this, but it wouldn't be a 'chemical process'.
But at the end of the day, they are offering the same tests that are currently available. If someone said to me we can now do the same old blood tests faster and cheaper, I would say that this is going to have very little impact on people's health. None of these tests are truly useful in prevention or screening unselected patients.
It does really seem that Theranos is a case of packaging the existing blood diagnostics and trying to capture a large part of the market. Exciting for investors I guess, but for patients or healthcare workers probably not so much. I wish they were doing something like cheap, easy and reliable comprehensive circulating DNA analysis, or proteomics, or something with immune cell function profiling...
I hope with all this cash they have a program for giving this technology to places that can't afford the usual lab infrastructure, that would be a very good thing.
Are you kidding? I'm not completely familiar here, but I'd love to see a comparison of the current SOC for time between measurements of each of these endpoints and then a clinical judgement that genuinely nothing interesting happens in these levels at timeframes shorter than SOC.
A person I'm familiar with has a severe endocrine disorder which has eliminated nearly all of his natural hormone production. Instead, he must self-administer all hormones in proper dosages. In order to survive this way he has become vastly more familiar with his own blood panel than most human beings ever would dare. On insurance dime he takes far more regular blood tests than you usually can get access to and understands, even very roughly, how events in his life, diet, exercise, stress, weather, etc begin to affect his blood chemistry and, subsequently, his life. In his, of course anecdotal, experience these are things that he cannot converse with doctors about because they're not able to speak confidently so far outside of SOC. It's not necessarily surprising stuff, but there's comparatively little real-world evidence to give it a bite.
Theranos could make similar analyses achievable for far more people. It opens an entirely new branch of clinical research---blood chemistry fluctuation at the level of perhaps up to multiple times per day over months at a reasonable cost in both testing and human time (given adequate distribution---Walgreens or, even, eventually, a home edition). You can't pretend that Phizer and GSK, known partners, aren't all about that.
> Exciting for investors I guess, but for patients or healthcare workers probably not so much.
Only if you assume SOC doesn't change under the influence of massively more available quantitative data.
He said "unselected patients". The point is, some random guy with no preexisting condition might not actually benefit from being able to measure shit in his blood more easily. It could mostly lead to false positives, stress, and unnecessary costs. The only way to know for sure would be to do a study on it... but Theranos doesn't seem to like doing studies.
Making blood tests faster and cheaper would definitely be a good thing, but it's possibly not a completely transformative thing.
Certainly happy to be proven wrong, but there is a risk of harming people (financially, psychologically and perhaps medically) if you convince them that relentless tracking of their serum sodium tells them something useful.
Their greatest claim, as least in the popular press, appears that they can do these tests on "a few drops of blood" (it appears to be about 0.5 ml from the image) as opposed to 10 ml or so. This claim is nothing special, as in fact almost all tests are eventually done on an aliquot no larger than a few microlitres that is withdrawn from the larger sample. As to speed of results, this would likely be linked to organizational efficiency rather than a technical innovation.
Further, there is no way all the tests shown on their website can be done with one 0.5 ml sample, as samples need to be collected into different anti-coagulants for different tests as some anticoagulants interfere with some tests but not other (this is why the doctor usually takes two or three samples into different tubes -- not because s/he needs 30 ml of blood).
They may however have improved automation, or a better run or cheaper service, in which case, good for them.
Whether it was due to the quantity of tests (7 tests) or the particular tests I ordered, I was quickly informed that I would be receiving a standard blood draw for my first visit. To my surprise, only 75-80% of patients receive the finger stick method, while the remaining 20-25% require a standard blood draw. The company’s goal to expand nationally relies upon the ability to integrate within the infrastructure of existing Walgreens’ drugstores, a feat made difficult by the need for a trained phlebotomist to serve a specific group of test orders. While the company currently offers more than 200 blood diagnostic tests, the internal assay development team is working to expand this menu to include over 1,000 of the most commonly ordered tests at their CLIA laboratory located in Palo Alto, CA.
In addition to my own blood draw, I stayed to observe the highly publicized finger stick, a staple of the start-up’s competitive appeal over established companies like LabCorp or Quest. It was here that the charm of Theranos became more obvious. Rather than a phlebotomist, a Walgreens pharmacy technician is easily able to perform the “nanotainer” blood collection steps, a testament to the simple and integrated workflow system developed by Theranos. Blood collection kits are provided to the technician, which include alcohol swabs, a warm compress, a finger stick, and a “nanotainer” (kept separately in a fridge). After scanning and labeling the sample tubes, the pharmacy tech begins the simple process as follows: apply warm compress to finger, clean the finger with alcohol swabs, massage blood towards the tip of the finger, stick the finger using one of 3 prick sizes (purple = small, pink = mid-size, blue = large), collect blood. The “nanotainer” which collects only a few drops of blood via capillary action, has continuously been an attractive marketing tool for Theranos, inflating the nano-sized tube with a larger than life persona. These samples are then stored in the fridge until a courier system can transport them to the CLIA certified lab in Palo Alto, where proprietary platforms allow the company to run traditional tests on 1/100th to 1/1000th of the ordinary volume. My results were emailed to my physician the following day.
http://decibio.com/blog/theranos-small-sample-big-opportunit...
I think you vastly under-estimate the impact of cheaper prices. We're not all sitting around with computers in our pockets because they're drastically different, we're sitting around with them because they're cheap.
Obviously there is skepticism on the applications, this is why she is giving all these talks about what this would enable.
I don't have any particular insight about what this kind of price drop will produce (though cheaper STD testing can't be a bad thing), but I think massive reductions in price do fundamentally change how things get done.
Without proper auditing and disclosure of development processes and the new implementation of the diagnostic tests on chips (rather than disposable test media) it raises the question of how can anyone be sure exactly what and what not the machines are testing and recording? Slot machines in Las Vegas have their code more closely audited and monitored than this technology.
This situation just yells out for government regulation, peer review and reverse engineering.
That's both the government.
When you're giving blood to some entity and you want to safeguard what happens to the result regulation won't get that for you, but independent oversight, sample anonimization at the source (insofar as this is possible with a blood sample, your DNA is your identity), decentralization of test locations (preferably as close to the source as possible), proven sample destruction and no keeping of data post sample destruction would be the way to go. Government oversight pretty much guarantees government access, the opposite of what you wish to achieve.
It will be very hard to create a service like this without a loophole that can be used to sneak such data out.
The best way in which this could be done is by having the tests made so foolproof that they can be done directly in the hospitals without centralized laboratories.
As for the board, you could say something similar about DropBox but I don't see them giving three letter agencies access to your account either without proper process and a court order.
Anything less could sink the company and given the amounts of money at stake here you can bet their competitors would love to do just that.
And who is to say that their competitors are not already acting in a similar manner anyway, they are after all processing about 100x more volume today.
And you wouldn't if they were served a NSL, which is very likely to have happened.
http://pathologyblawg.com/pathology-news/new-yorker-theranos...
The famous Forbes-List estimates her wealth to be 4.5b, because she owns half of a company that has raised 200m. If she is somehow reasonable with the finances of her company, it is likely that her actual bank-account (or real-estate etc) is well below 10m - which is a hugh difference to all the old-money people far below her in the list.