Hacker News Reader

Comment by Exmoor | Hacker News Reader

>Nearly one third of 200 Chelsea residents who gave a drop of blood to researchers on the street this week tested positive for antibodies linked to COVID-19, a startling indication of how widespread infections have been in the densely populated city.

>The doctors used a diagnostic device made by BioMedomics, of Morrisville, N.C., to analyze drops of blood. It resembled an over-the-counter pregnancy test and generated results on the street in about 10 minutes.

This is the only recent testing I've seen that appears to be anything remotely resembling random. The sample size is also not incredibly small. That said, this is the first I've heard that there's a handheld device which can deliver an antibody test in 10 minutes and being completely out of that industry I have to wonder how likely it is that it's delivering a high level of false-positives?

View on HN

This doesn't seem that random - it is sampling from people on the street. So it's going to miss people who aren't going out onto the street. And people who are going out are more likely to catch something than those who aren't.

The previous tests I'd seen (I hadn't read about the Los Angeles test results posted today yet) had been fairly extremely not random. For example the homeless shelter in Massachusetts and pregnant women in New York. That's why I used the phrase "remotely resembling random."

Because you quoted the article using BioMedomics' antibody diagnostic test, I will quote the accuracy from their site[0]:

> In order to test the detection sensitivity and specificity of the COVID-19 IgG-IgM combined antibody test, blood samples were collected from COVID-19 patients from multiple hospitals and Chinese CDC laboratories. The tests were done separately at each site. A total of 525 cases were tested: 397 (positive) clinically confirmed (including PCR test) SARS-CoV-2-infected patients and 128 non- SARS-CoV-2-infected patients (128 negative). The testing results of vein blood without viral inactivation were summarized in the Table 1. Of the 397 blood sample from SARS-CoV-2-infected patients, 352 tested positive, resulting in a sensitivity of 88.66%. Twelve of the blood samples from the 128 non-SARS-CoV-2 infection patients tested positive, generating a specificity of 90.63%.

[0]: https://www.biomedomics.com/products/infectious-disease/covi...

Specificity of 90% seems really low? Does that mean almost 1 in 10 would test positive even if nobody had it?

Yes, it's 10% false positive. But it's not "really low" in the sense that it's typical for antibody tests. Which is one reason not to do wide-spread testing.

Or you could flip that around and make testing so plentiful everyone is tested 3x (perhaps by independent labs), so the FP rate is more like...

1/10 * 1/10 * 1/10 (1/1000)

No good reason for test errors to be uncorrelated, ie. quite possible same people who trigger false positives on the first test would be false positives on the second etc.

Sure, there will probably be some correlation. Depends heavily on how the 3x testing is designed.

T1. Drive up saliva swab by nurse, for serology test.

T2. Re-deploy census peeps to get samples (and census info) from entire neighborhoods.

T3. 23andMe-like home kit express mail to lab

The odds of people getting a FP on all three might not be 1 in 1000, but it probably isn't 1 in 10.

Anyway, as it is, leading 1 in 10 people to falsely believe they are immune, isn't much better than having no test at all.

Assuming false positives aren't correlated even with independently manufactured tests seems like a terrible idea to me, knowing little about medicine.

Sounds like you had a bad experience once, with 'double checking'.

This assumes that the cause for false positive is in the test execution and not a contamination in the sample itself.

This would require that false positives are truly random and not influenced by sample-specific factors. Is this the case? In antibody testing, I would imagine that false positives result from cross-reactivity instead of random chance.

Sure, there are certainly protocols that could be made that would almost ensure FP is due to sample factors.... or even test related factors that would maximize 3x FP (same lab, same lab tech, same machine, same day, back to back, same reagent stock).

We shouldn't adopt those protocols.

That's a correct understanding of specificity.

That would imply the correct rate is closer to 22% than 30%, as of the 64 positives out of 200, we would expect roughly 20 of those to be false positives. (So the remaining 44/200=>22%).

Article says there are 40k residents and 39 deaths.

If 22% of 40k are infected => 8800 infected.

37/8800 implies .42% IFR, broadly consistent with figures elsewhere, especially given that there is a lag time for deaths. (Which will cause the IFR to increase.)

Any bias due to selection etc would basically decrease the denominator on that calculation, increasing the IFR.

(This is all just back of napkin, based on numbers in the article, please check my calculations.)

A 10% false-positive rate is awful. If the actual incidence is 2%, a test with that error rate could overstate the results at about 12%. That combined with the sampling bias could easily result in overstated numbers.

Randomized household sampling would be far preferable. That would obviously take much more time and would expose testers and the household to more risk. But without good methods, research like this and the surveys conducted in Santa Clara and LA counties are potentially worst than useless since they have the potential of misinforming policymakers and the public.

So interestingly, those results from the research paper with more details ( https://www.biomedomics.com/?fldl=3050 and also https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25727 ) credit a different manufacturer called Jiangsu Medomics Medical Technologies. I wonder if that's their subsidiary, or if they're buying the tests from them.

This specificity issue is bothering me.

This 'medical' meaning 'very important' - so if there's a test with 90% accuracy, why on heaven's earth do we not simply run the test 3 times to get 'considerably greater accuracy'?

10% error is so large it's tough to make heads or tails of the data?

Does someone know if this makes sense i.e. if we can simply run the test 3 times and get better data.

Secondly - why are these health authorities not doing proper, state-wide tests?

Here we are with an economy in meltdown, trying to 'model model model' with a 10 Trillion dollar economy wouldn't it make sense for at least ONE (or a few) freaking comprehensive tests that give us some good data?

It has shown to have a 2% false positive rate lower than the 10-15% false positive rate of the general test so its the most accurate test we have.

https://abc7news.com/coronavirus-test-free-testing-update-ac...

Reply on news.ycombinator.com