Hacker News Reader

Comment by JTon | Hacker News Reader

A take it or leave it anecdote for you: I was chatting with an actuary working in life insurance field who is heavily involved in the pandemic industry groups. The actuary told me the antibody test is considered unreliable and has as high as 50% false negative rate.

View on HN

Anecdotes aren't worth much in this area. There's no conspiracy of silence about problems. We can use actual science. They actually test the quality of these tests. Some are better than others.

For example, the Abbott test has a surprising accuracy [1] "Researchers at the University of Washington School of Medicine found Abbott’s test had a specificity rate of 99.9% and a sensitivity rate of 100%, suggesting very few chances of incorrectly diagnosing a healthy person with the infection and no false negatives."

1. https://www.cnbc.com/2020/05/08/study-suggests-abbott-covid-...

Independent testing of the Abbott assay found lower sensitivity, 94%. The blood samples used to establish sensitivity are also from patients who got sick enough to seek medical care, who likely have higher levels of antibodies than milder cases. It's still one of the better tests, just not as perfect as their marketing numbers imply.

https://academic.oup.com/clinchem/advance-article/doi/10.109...

Look at the 95% confidence intervals on those results.

I wouldn't take a study with N=256 (103 positive, 153 negative) seriously in claiming the specificity is a couple percentage points lower than the manufacturer's standard. That's simply too low of a cohort to draw that conclusion.

Even the conclusion from that study you linked was actually that the test wasn't that accurate in the first 14 days of infection (which is already known for ab tests), not that test specificity is lower.

First, the original question was about false negatives, and therefore about the sensitivity (positive rate on true positives), not the specificity (negative rate on true negatives). So perhaps you wrote specificity by mistake? By definition, the sensitivity can be calculated only from true positive cases, so the relevant N is just the positives. There were 103 positive samples from 48 patients, and the independent paper did their math using N = 48.

Abbott's marketing referred to a study of 125 patients confirmed positive by PCR, linked below. So it's more, but not dramatically so. The affiliated study doesn't report a confidence interval, but the binomial 95% confidence interval is [97.1%, 100%]. That barely overlaps with the independent study's [82.8%, 98.7%]. So perhaps the affiliated researchers just got lucky and the independent ones got unlucky, or perhaps too many of the independent paper's >=14 day samples were <17 days and they should have waited longer; though depending how cynical your priors are you might also conclude the groups were different (not to allege any misconduct, just to say there's a lot of reasonable judgment in what patients you select that could push the result around by this small amount).

Regardless, the sensitivity of the test is definitely not an unqualified 100%. Even disregarding the independent study completely, that would still be a somewhat misleading claim, given the small sample size in the affiliated study and that patients with mild illness probably develop lower titers than the average patient who sought medical care over the last few months.

https://www.medrxiv.org/content/10.1101/2020.04.27.20082362v...

I'm still suspicious there's something going on that we don't understand with those tests. Someone I know was seriously ill for several weeks starting in late March with 9 out of the 11 listed symptoms for COVID-19. She eventually recovered and then tested negative on the Abbott SARS-CoV-2 IgG antibody assay. We can't definitively rule out some other virus but based on the timing and symptoms what else could it have been? It certainly didn't present like influenza.

I think the public is simply unaware of how bad the "regular" flu can be... it's a long tail but every year otherwise apparently young healthy people die of the flu. Not to mention confounding factors of latent conditions and concurrent diseases.

The implication of conspiracy theories is that the Chinese government:

* Perfectly covered up the initial outbreak

* Took no further action

* Utterly failed to cover the "second leak" that led to the pandemic

Seems unlikely?

An alternative theory is social media now disproportionately focuses on the regular bell-curve extremes of our normal flu season.

>I think the public is simply unaware of how bad the "regular" flu can be.

It seems that everyone says they have the flu when they become unwell, until they actually get the flu. The difference is quite clear when you get it.

My wife and I visited Iceland last July. 5 days of the trip were spent trekking in the backcountry with a group from 4 different continents. When we got home, my wife came down with a terrible case of the flu that led to pneumonia. She never needed to be hospitalized but it was close for a couple days.

Eventually, it spread to my son and myself. It's the sickest I've ever been in my 40+ years of life. We didn't return to anything resembling normal until early October. It can't have been COVID but there must have been at least one really nasty flu bug going around last year.

It always annoys me that anytime anyone has a minor cold, it's "I had the flu". That lack of understanding is why more people don't get flu shots.

Occam's razor says she had the flu and not Covid-19.

Influenza isn't one thing and the flu can have severe symptoms. Swine flu is also still going around. My wife's coworker (large tech company in the Bay Area) was positive for swine flu around March.

> Occam's razor says she had the flu and not Covid-19.

And, in most places, test positivity rates.

For months in my area covid-19 tests were only available for people with symptoms matching covid-19 - and those people were still testing 95%+ negative for covid-19.

Perhaps, but I've had the flu several times myself and seen many cases in others. This sure didn't look like a flu. I should add that in this particular case the patient's doctor diagnosed her with Covid-19 on a video call based on symptoms. And the patient had a current flu immunization (I know the flu shot isn't completely effective but even when it doesn't exactly match the viral strain it usually mitigates the symptoms).

I also had weird flus this year and last fall. Still tested negative. There are hundreds of viruses and many mutations that give similar symptoms. It's not unusual that you get unusual one for you this one time

It's always possible it was something else, but lack of (testable) antibodies does not rule out COVID-19 infection, as they are not always developed and/or expressed after some time. Seems like that may even be common.

Since T-cell immunity is rather hard to test, you may never know.

I was also pretty sick in early March but not sick enough to get a test then (they were in short supply of course). I recently got the Abbott blood-based anti-body test and I was quite surprised that I hadn't had covid-19 either.

COVID test accuracy is very similar to condom effectiveness: they're both high in theory, but surprisingly low in practice, not because of the science but user error. I've seen estimates as high as 10-30% for false negatives, largely due to poor swabbing practices:

https://www.healthline.com/health-news/false-negatives-covid...

First, if you're talking about swabbing, then I think you're talking about the test to see whether a person is currently infected with the virus. That's what the article you linked was regarding.

The antibody tests are a different animal. They test to see whether you might have been infected in the past, by looking for antibodies in your blood. So these are blood tests, and, I expect, have less chance for user error.

But also, I think lots of covid antibody tests are not high accuracy even "in theory". It seems the FDA relaxed oversight on companies producing antibody tests: ". . . after being criticized for the fumbled rollout of diagnostic tests during the start of a global pandemic, the FDA swung hard in the other direction, waiving its usual requirements and letting firms rush self-validated [antibody] tests into the market."[1]

So there are antibody tests from over 200 companies out there. Some of these antibody tests are quite accurate, high sensitivity and specificity. Others, not so much.

[1] https://khn.org/news/hype-collides-with-science-as-fda-tries...

The Abbott test mentioned above is blood drawn from serum (not a fingerprick blood test!).

This is not the nasal swab test that sometimes misses positive people, as you mentioned.

To avoid confusion over this issue: There is an Abbott nasal swab test, that gives results 15 minutes after being swabbed, and without "special equipment". That specific Abbott test (the instant nasal swab) has experienced dubious results in practice, even compared to more standard nasal swab tests.

That's not the sort of test under discussion here. The Abbot test and other antibody tests are blood based. I'm sure there are things in the pipeline that can go wrong, but they're much less susceptible to error.

What is important is that antibodies are a sufficient but not necessary indication of past infection. There seem to be plenty of people who don't develop them when exposed, possibly due to effective T-cell immunity.

FWIW I think most if not all of the antibody tests are done on blood

There are different tests out there, many are pretty bad. Some are pretty good, for example Abbott's clears 90% in both specificity and sensitivity. That said, given a low prevalence of the disease, this is actually fairly likely to give false positives. 93.8% sensitivity suggests the false negative rate should be pretty low with that test.

https://www.360dx.com/immunoassays/study-finds-abbott-corona...

Not to mention the PCR tests and the antibody tests got combined into "total cases" in several data sets, mucking the numbers:

https://www.theatlantic.com/health/archive/2020/05/cdc-and-s...

Neither being an actuary or insurance adjuster makes you an expert on clinical lab tests.

My guess is the person you were chatting with heard some hyperbolic statistic about ELIZA tests and is repeating it.

Not so anecdotal: There is an Oncology professor in the UK who tested his staff of 200 and got ~10% for antibodies. A month later he expected 30%, but still got around 10% (https://unherd.com/thepost/professor-karol-sikora-fear-is-mo...)

There is a chance antibodies for this simply do not last very long, or that many people fight it off before it reaches the adaptive immune system and requires antibodies (there are many layers that have evolved in our immune systems).

There are many other viruses that don't produce long lasting immunity as well. Hepatitis B vaccines require 3 shots within a few months and even then, the take rate for it is only around 60%.

That link is rather dubious. eg "When the history books are written, the fear will have done much more damage than the virus". This is easy to say when you're one able to read the history book. It's all going to depend on what ones acceptable death rate is, and that gets dark pretty fast.

The author also has a history of controversy which definitely makes me want to take anything with a grain of salt:

https://en.wikipedia.org/wiki/Karol_Sikora

There’s likely a reason why that’s on a conservative business’s vanity site and not a scientific journal.

Reply on news.ycombinator.com