Theranos and the Blood-Testing Delusion | Hacker News Reader

Theranos and the Blood-Testing Delusion

bloombergview.com

76 pointstokenadult54 comments

This article points out the major issue in blood testing: it's not a panacea. So often, particularly outside the life-sciences press, the idea that we'll be able to test for everything and that it will be useful gets shouted from the rafters.

The problem is, it's not that simple. Biological variation and lab procedure variation and just plain dumb bad luck makes it so that any lab test should only be considered a single datapoint. To be taken seriously, but not to be taken as "the answer" in and of itself.

Technology can solve the lab procedure variation issue, but can't solve biological variation. Maybe it can alleviate the biological variation problem by making it feasible to test more frequently. However as you multiply the things you're measuring and multiply the frequency of testing you might end up testing too many hypotheses too frequently for your statistics to be reliable.

I'd love to have more tests more frequently. I work in medical informatics after all. But We need to be cautious about what will come out of it and not over-promise.

Much as we'd like it to be true, big squishy bags of water with bits of hardened calcium in them are not computers. Anything biological is hugely more noisy and far less deeply understood than anything we deal with in information technology.

Solution appears to revolve around getting more data. There are inherent limits in taking something out of the body to destructively test it. Remote imaging what can be imaged, and then relying on vessel-implanted data collectors may be the fruit the vision is in search of.

More data is more better. Imaging is nice, but it's really hard to do biochemical tests via imaging. It can be done but hasn't been much so far outside the lab. Implanted samplers are cool, but they get fouled up pretty easily and pose an infection risk, so there's still some progress to be made there.

I'm just starting to go down the blood testing route for myself and my startup (we're going to pay for before/after blood tests for certain supplement reviews), and I find it hugely fascinating. Others do too, and because of that, there's going to be continued growth.

I am absolutely convinced that we'll someday get to where Theranos wants to go - possibly even better - but it's going to take more time and it's going to have to come far more slowly. Costs need to get down a lot more.

You can't advertise and get funding for step 100 when we're still on step 3. This was their fatal sin.

Either way, given the interest I'm seeing in our program, I know the future will push forward. We're clearly still on the very beginning part of the adoption curve with this one -- way earlier than Theranos led us to believe -- but someday I think we'll have the data we want, and in continuous fashion too.

It's still all very reactive stuff though -- until we can read what our nervous system is communicating to the body, we're just playing an ugly, rudimentary form of catch-up. I don't see that really changing in my lifetime though.

I think you're right, it's someday possible, but it's quite early days. The danger of over-promising in medicine is that when you fall flat, in the best case you badly damage the pipeline of people trying to solve it, and in the worst case you injure people.

> The problem is, it's not that simple. Biological variation and lab procedure variation and just plain dumb bad luck makes it so that any lab test should only be considered a single datapoint. To be taken seriously, but not to be taken as "the answer" in and of itself.

And even if the results are accurate it's often not useful. See the amount of over-diagnosis and over-treatment that results from too much testing.

Indeed, and never forget the false positive paradox:

https://en.wikipedia.org/wiki/False_positive_paradox

I'm consistently surprised at the low bar for due diligence in startup funding. In my personal experience, it goes something like "send us a business plan, pitch deck, three customers (that you handpick) for us to talk to, the legal paperwork, and maybe some bank records but not usually". I always assumed investors required credentials to Google Analytics or Mixpanel or iTunes analytics or whatever to actually verify that the usage numbers are what founders say they are, yet I've never had an investor do that or even heard of it being done. It's completely baffling to me that Theranos can raise ~$700M w/o any third party testing or diligence.

This article suggest it's because Holmes is "enthralling" and investors had an "exaggerated faith in the power of medical testing". Faith and charisma are very abstract concepts, but data and testing aren't, so why rely on the former instead of the latter? As Deming said, "In God we trust; all others must bring data."

This is business!

http://www.techaccent.com/bill-gates-son-in-law-joke/

I do diligence for a living. It's just not economical to do a full diligence on every investment when you're investment thesis is a portfolio approach.

Agreed. Also there are typically legal repercussions for misrepresenting your business that are outlined in deal terms.

Exactly. If things go south, legal can uncover and if lies were presented then things can turn criminal as theranos now knows.

But that will not recover your investment.

It can depending on how much capital or assets are still around.

Jail is also a pretty decent incentive to make people avoid overtly lying. Unfortunately its disproportional used on the poor and minorities instead of on professionals.

Those are usually covered under "reps and warranties". From what I understand, even if you do "lie" (no one really lies, they just bend the truth...if you do legitimately lie, it's fraud and it's a big deal) there's basically insurance against it (formerly called "representations and warranties"). No VC I know of ever purchases it, but PE do from time to time.

What is the cost for something like that? Any way you can give an idea w/o giving away specifics you don't want to disclose?

Typically there is technology, legal, and financial diligence. Each costing anywhere between $30-100k each.

Wow. How?

Some of this is actually the sign of a bubble in startup investing. When the market is tough, investors typically do much more diligence (they are more careful and they have more leverage). When the market is superheated, there's so much money chasing any deal that looks remotely interesting that any investor who wants to do real diligence is basically shut out by the investors who are willing to put up a check without it. FOMO overcomes the duty of care.

It's like buying a house in a hot market; cash offers with no contingencies win out over "smart" buyers who want to know what they're getting.

Was this like that? Because it seems like Theranos had a pretty tough time finding investors, or at least reputable ones. GV laughed them off. As did all the life-sciences firms.

It may be a cousin to it. In this case, perhaps second-tier investors saw it as an opportunity to get into a hot sector, with some comfort (and reduced diligence) because there were investors like Tim Draper involved.

> at the low bar for due diligence in startup funding

In the case of Theranos you also have how she was well know by Tim Draper (DFJ investor) apparently friends with his daughter. So he must have been smitten by her intellect possibly in comparison to his own daughter (my speculation but wouldn't surprise me).

http://www.mercurynews.com/michelle-quinn/ci_26147649/quinn-...

"Tim Draper, the venture capitalist, said Holmes, who was friends with his daughter growing up, is the first entrepreneur he knows who kept quiet about her business for so long "so that the competition wouldn't get a chance to start."

"She had a winner and knew it," he said. His firm DFJ Venture was the first to invest."

In a fair world, Holmes would have to have been an absolutely remarkable 19 year old to get what she got. I mean, like Leonardo da Vinci level, once in a civilization type amazing.

It turns out she's not a very remarkable person today, and wasn't as a 19 year old college dropout either.

She's always come across as a bit of an odd duck, even when the bubble hadn't burst [1]. People (investors) likely read into her what they wanted and the myth of the genius who's too smart for traditional paths provided a convenient framework to hang a cool story off of and allowed laziness to creep in. Her impressive personal connections were fuel.

One thing to watch and keep in mind as Theranos come unwound [2]

1 - https://www.youtube.com/watch?v=UiNFXcI9Rb8

2 - https://www.youtube.com/watch?v=hLTAFbKbC8w

Right, there seems to be a big problem with pattern matching, i.e. "quirky Harvard dropout" and simply associating that with success.

This article isn't really about Theranos, so much as it is about the unreliability of blood testing as a diagnostic technique even when the tests are working properly.

True, but I'd say that the unreliability of blood testing as a diagnostic technique is extremely pertinent information for investors to have when investing in a company that is built on blood testing as a diagnostic technique. My point is just that it's surprising that the journalist for this story appears to have spent more time researching the subject for this article than the investors putting in ~$700M.

Again: the article isn't about whether Theranos was a good investment. It's about whether, even if Theranos had worked perfectly, convenient blood testing would have improved medical outcomes.

Yes, you've correctly identified what the article is about, and I've never disagreed or said "this article is about Theranos as an investment". I still think it's interesting to think about the article in the context of due diligence in startup investing.

>> That way she could see how many diseases were caught early, how many diseases were missed, and how many times false positives occurred

I don't understand why this article continually mixes one thing tests do(measuring physical quantities) with another(predicting disease) ? The way to evaluate theranos is whether they measure the physical quantities correctly(and it appears they are not), not with the complexity of predicting disease from those numbers. That's a job for other parts of the medical establishment.

You can catch many things early, if you test your blood often. Cancer is the big one.

Provided you can measure it accurately first.

I think you're missing the point. The point is there are two parts: 1) accurately measuring what's in the blood and 2) making an accurate prediction of future outcomes based on the measurements of what's in the blood. You can fail at one and succeed at the other, but it's not really useful unless you can do both.

Assuming this is true (which is a stretch) this is tricky business - you are assuming detecting all types of cancer in asymptomatic individuals has no downsides. In reality it has major downsides, this is the reason why the the U.S. Preventive Services Task Force no longer recommends the PSA antigen test be used for routine population wide screening.

http://fivethirtyeight.com/features/the-case-against-early-c...

That's early testing, specifically "average-risk men aged 55 to 69 years".

E.g. my 82 year old father would be dead or severely suffering today if his routine, over 69 years of age PSA tests hadn't prompted his doctor to order a biopsy 5 years ago.

Your father might be dead or suffering today. It is impossible to be certain about the natural course of a disease after you intervene. I'm glad your father is doing well, of course, but we should be cautious: stories like this one tend to bias people towards treatment with what is technically a false representation of facts. I hate to seem pedantic, but accuracy is crucial in discussions like this.

Given the Gleason score, let's just say would/might -> most likely.

I don't doubt that you (or whoever) had additional information and made a sound decision.

Point being simply that someone reading your original comment (or such comments generally) doesn't have the full context and might not realize it. I'm not making a point about your situation, but about how the internet is often misused by people seeking to make their own decisions. A subject that fascinates me.

Well, all that's after the fact 20/20 hindsight.

Initially, all he and his doctor knew was that he was 77 years old and his routine, approved by the powers that be, PSA test results took a worrisome turn, bad enough that a biopsy was indicated.

My father's case only supports the Official Word, that starting at age 70 you should get PSA tests. WRT to people making their own decisions, I highly advise that, if you're 70 or older, to get regular PSA tests!

>My father's case only supports the Official Word, that starting at age 70 you should get PSA tests. WRT to people making their own decisions, I highly advise that, if you're 70 or older, to get regular PSA tests!

I highly advise you don't take medical advice from hga on Hacker News based on one anecdote. The Official Word from the U.S. Preventive Services Task Force is "The U.S. Preventive Services Task Force (USPSTF) recommends against prostate-specific antigen (PSA)-based screening for prostate cancer."

http://www.uspreventiveservicestaskforce.org/Page/Document/U...

https://www.sciencebasedmedicine.org/screening-for-disease-i...

Surgery for prostate cancer has some very serious side effects...

Before the fact, we know that cancer testing does not reduce mortality, with the possible exception of cervical cancer. PSA tests are the worst offenders.

Why cancer screening has never been shown to “save lives”—and what we can do about it

http://www.bmj.com/content/352/bmj.h6080

I think a lot of the blame falls on the hype machine from Silicon Valley these days. SV has mastered manipulation of the press, and that's a power that can be used just as much for bad as it can for good. The drive to have the next celebrity CEO as well as being the talk of the town has driven this media craze. Relying on the media to drive celebrity rather than accomplishments and engineering chops is what changes SV back into any other mundane cronyistic industry.

Looking back at the Theranos hype, it was exactly the phrases that SV loves to hear, but without any of the data to back it up. That can work in software to some degree, where you can just build what you need, and it's just an engineering problem. You can't do that with science; nature is not as malleable as code.

One thing about Theranos that seems different from most SV hype stories is the insane level of focus on the founder. Every piece that reports on Theranos spends more time on her than it does the product. I'm speaking from minimal experience here (my start-up credentials right now extend to one short-lived company), but I think normally VCs aren't willing to "fund the founder" vs. "fund the product" outside of the very early stages of a start-up. Theranos has been around for a decade and the fact Mrs. Holmes is a Stanford dropout is still nearly in every single article about the company. Contrast this to Mark Zuckerberg and his former association with Harvard is nearly never mentioned anymore (In articles dealing with Facebook).

That is an interesting call-out. I think originally, that it was a deliberate part of the Theranos story, trying to piggyback on the life stories of the likes of Gates, Jobs and Zuckerberg. Now it's beginning to backfire.

Facebook never had such a personality centric story past the panache of coming out of Harvard and the Ivy League.

When we're talking about Facebook, we have a zillion concrete things to talk about, the services they provide, their open source and hardware, their refusal to play ball with Jobs' attempt to turn SV into a company town, etc. etc. All sorts of things one associates with successful companies, which here I define as actually doing stuff, ignoring the minor detail of making money off of it.

Theranos is not, and has never been a successful company, aside from hype and attracting investment. Their simple inability to properly and legally run a basic, conventional diagnostics lab suggests its not even in their DNA, even before one notes that they've only been able to get one finger stick test approved by the FDA.

Silicon Valley is already circling the wagons to deflect blame:

http://www.nytimes.com/2016/04/27/opinion/dont-blame-silicon...

I wonder how this bodes for paper diagnostics? http://www2.technologyreview.com/news/412187/tr10-paper-diag...

Maybe the solution isn't higher one-shot accuracy, as with nanotainers(tm). Maybe it is higher frequency? The idea of creating blood diagnostics for pennies is so compelling.

Hate to see diagnostic research suffer from one bad investment. Am reminded how Minsky-Papert's book on Perceptrons created a winter for neural net research.

Theranos has a really curious set of investors and board members: http://www.techinsider.io/theranos-board-of-directors-2015-1...

Maybe it didn't matter that much whether the tests actually worked.

If you want frequent testing, and this is not my idea, but that of someone who's been doing serious biology for decades, the most obvious was is to see what we excrete, something that's part of a toilet.

Needless to say, this won't make for a glamorous startup....

Interesting. Any links to diagnostics that can be run from fecal samples compared to blood samples?

Don't forget urine, the kidneys give quite a view into one's bloodstream.

Don't have any links, it's just something that is said to likely be fruitful.

The thesis here is that having a personalized healthy baseline for many bookmakers will be far more powerful than comparing a diagnostic against population based averages.

Also that higher time frequency I the data will also be revealing.

There is some research in support of this idea[1] but it remains highly speculative.

[1]http://med.stanford.edu/news/all-news/2012/03/revolution-in-...

The thing that amazes me is that Theramnos was valued at 9M. Tabcorp's market cap is $13B on revenues of $6B; Quest is $11B on revenues of $7.6B.

With a $9B market cap already, even if Theramnos were able to take 100% of the market and have a higher margin product it would be at best a 2.5X return for its investors -- hardly a "10 bagger"!

I never "got" it because of those economics and the crazy science of tiny samples.

>> He used his sister’s breast-cancer markers as an example... her risk may not have changed at all, the variation being the result of natural biological fluctuations and inherent uncertainty in the measurement. >>

That natural biological fluctuation is exactly what pinprick blood testing could help with, because (in concept) the blood test could be delivered much more frequently, and measure the true variability.

I don't see why pinprick testing would help; it's not like a regular draw is that unpleasant (I went through a period of my life where I had two cylinders drawn every week).

The opposite problem exists with the tiny samples: blood like any other fluid, is not at all points identical at all times. You can easily demonstrate this for yourself: go to the drug store and buy a cheap glucose meter and take a bunch of samples just around the same finger, much less different fingers. You'll see a surprising level of variation. The larger samples of a conventional draw are shown to vary less.

They still vary (as the line you quote says) but less than the highly local samples.

Reply on news.ycombinator.com