Nearly a third of blood samples taken in Chelsea show exposure to coronavirus | Hacker News Reader

Nearly a third of blood samples taken in Chelsea show exposure to coronavirus

bostonglobe.com

82 pointsrpiguy74 comments

Can we change the title?

Article title is "Nearly a third of 200 blood samples taken in Chelsea show exposure to coronavirus"

The sample is clearly not random

Your never going to have a completely random test. Your randomness comes from you picking for an area and the people who chose to do the test. Saying otherwise is being pedantic to your detriment.

It’s normal to randomly sample multiple locations. Random samples from even 10 random locations are much more useful.

Not really... more data can't fix sampling bias.

It’s not a question of ‘more data’ as I could take 50 or 50,000 samples from a single person and I gain nothing about the overall population.

Similarly if I sample 50 or 50,000 people from a single city I learn nothing about people outside of that city. But, by sampling N random locations I at worst have N samples to work with.

PS: At the extreme, if I sample every single member of a population then ‘more data’ solved any bias problems. Smaller samples are subject to a wider range of biases.

Not if you are trying to figure out the best decision for a specific city/state

You can choose to offer people some incentive to take the test. If you offered $50 per person you would probably eliminate most sampling bias.

You're more likely to attract people of lower incomes, which we already suspect are more vulnerable to Covid given the kinds of jobs they have.

> Your never going to have a completely random test.

You are not, but on the full spectrum of somewhat random selection methods, talking to people on the street is about as biased as you can get.

Yep.. I saw a similar headline claiming LA infections are 55x the reported rate. I think the math is likely simpler, and the infection is somewhere between 3x-5x (10x tops). I've read a few reports saying about 50% of people are asymptomatic, 30% show symptoms that aren't bad enough for hospitalization and 20% require hospitalization. I can't find the source I got it from but I believe it was a Chinese study

If anyone has good data on the asymptomatic proportion, I'd appreciate a link. I keep seeing 50% (or sometimes 'up to 50%'), but to the extent that I've looked at actual sources, it seems that people are conflating presymptomatic positive tests with lastingly asymptomatic infections.

Based on several studies using different methods, the death rate (infection fatality rate, or IFR) is likely around 0.5%

So if you go back the average time from infection to death (2-3 weeks) you can get a sense what the ratio was in your area back then.

[1]: https://mothership.sg/2020/03/iceland-covid-19/ (I did the work on the side, extrapolated to about 0.2%-0.5%)

[2]: https://www.land.nrw/sites/default/files/asset/document/zwis... (Germany study, 0.37%, decent sampling method)

[3]: https://www.ncbi.nlm.nih.gov/pubmed/32234121 (Diamond princess ship, 0.5%)

[4]: https://www.medrxiv.org/content/10.1101/2020.04.14.20062463v... (Santa Clara county study, 0.15% - 0.3% if you do the math). Methodological problems with the method of sampling that might increase the number of positives and the test specificity issues might increase false positives, but probably still in the ballpark

Dimond Princess had 14 deaths out of ~700 infected or a 2% fatality rate though that number has slowly climbed over time. As they tested everyone initially finding a large number of asymptotic cases that’s likely extremely accurate.

The difference is the population was more heathy than the general population, and while older on average, it had a lower percentage of people 95+ years old.

This also came out today. http://publichealth.lacounty.gov/phcommon/public/media/media... We really have no idea what the IFR is, just that is probably in the .1% to 1% range. Also, it takes around 2 weeks to develop antibodies, so the tests are showing how many people had covid 2 weeks before testing.

That antibody timeline was an estimate derived from data obtained from the "original" SARS virus, but it isn't accurate with the SARS-CoV-2 virus. IgM antibodies are present within hours of the onset of symptoms, and IgG antibodies after a few days.

Source?

Sure, find any manufacturer of rapid antibody test kits and look at their documentation for a nice trailhead into plenty of literature about antibody response times with this virus.

“Public health experts already knew Chelsea had the state’s highest rate of confirmed COVID-19 cases and that the actual rate was probably higher”

Should be 1/3 + in town with highest confirmed case rate. It may be a random sample within that town but it’s explicitly not representative of MA

In addition, the wording "test positive for Covid antibodies" may lead people to think that 30% of the population in MA is immune to Covid19. It obfuscates the fact that the mere presence of antibodies does not imply immunity. There has to be a sufficient amount of antibodies in order to successfully fight off reinfection (which could be worse than the initial infection) ...

Ok, changed now. Submitted title was '30% of random sample test positive for Covid antibodies in Massachusetts'

Hmm, I wonder if they attempt to control for bias. It sounds as though they were asking random passersby if they wanted a COVID-19 antibody test. If so, it seems that the sample group could be biased toward including more people who suspected that they may have been infected (previously or currently) than a truly random sample.

Also biased because they are sampling people who are out and about. People who are out and about likely come into contact with more people than people who are staying at home and rarely going out.

Out, about, and willing to speak to a stranger, which I think is the biggest bias. Pretty much everyone I know goes on the occasional walk, but we all stay the fuck away from other people.

My Hollywood movie plot theory is that Covid affects inhibitions.

It’s certainly not a hollywood movie—for the flu, it’s already the case!

Change in Human Social Behavior in Response to a Common Vaccine

https://pubmed.ncbi.nlm.nih.gov/20816312/

Conclusions: These results show that there is an immediate active behavioral response to infection before the expected onset of symptoms or sickness behavior.

You might enjoy David Cronenberg's film Rabid.

Except this aligns with other general sampling. You're not going to be able to sample people with consent.

Not like you can force people to test so it's impossible to get full randomness there.

>Nearly one third of 200 Chelsea residents who gave a drop of blood to researchers on the street this week tested positive for antibodies linked to COVID-19, a startling indication of how widespread infections have been in the densely populated city.

>The doctors used a diagnostic device made by BioMedomics, of Morrisville, N.C., to analyze drops of blood. It resembled an over-the-counter pregnancy test and generated results on the street in about 10 minutes.

This is the only recent testing I've seen that appears to be anything remotely resembling random. The sample size is also not incredibly small. That said, this is the first I've heard that there's a handheld device which can deliver an antibody test in 10 minutes and being completely out of that industry I have to wonder how likely it is that it's delivering a high level of false-positives?

This doesn't seem that random - it is sampling from people on the street. So it's going to miss people who aren't going out onto the street. And people who are going out are more likely to catch something than those who aren't.

The previous tests I'd seen (I hadn't read about the Los Angeles test results posted today yet) had been fairly extremely not random. For example the homeless shelter in Massachusetts and pregnant women in New York. That's why I used the phrase "remotely resembling random."

Because you quoted the article using BioMedomics' antibody diagnostic test, I will quote the accuracy from their site[0]:

> In order to test the detection sensitivity and specificity of the COVID-19 IgG-IgM combined antibody test, blood samples were collected from COVID-19 patients from multiple hospitals and Chinese CDC laboratories. The tests were done separately at each site. A total of 525 cases were tested: 397 (positive) clinically confirmed (including PCR test) SARS-CoV-2-infected patients and 128 non- SARS-CoV-2-infected patients (128 negative). The testing results of vein blood without viral inactivation were summarized in the Table 1. Of the 397 blood sample from SARS-CoV-2-infected patients, 352 tested positive, resulting in a sensitivity of 88.66%. Twelve of the blood samples from the 128 non-SARS-CoV-2 infection patients tested positive, generating a specificity of 90.63%.

[0]: https://www.biomedomics.com/products/infectious-disease/covi...

Specificity of 90% seems really low? Does that mean almost 1 in 10 would test positive even if nobody had it?

Yes, it's 10% false positive. But it's not "really low" in the sense that it's typical for antibody tests. Which is one reason not to do wide-spread testing.

Or you could flip that around and make testing so plentiful everyone is tested 3x (perhaps by independent labs), so the FP rate is more like...

1/10 * 1/10 * 1/10 (1/1000)

No good reason for test errors to be uncorrelated, ie. quite possible same people who trigger false positives on the first test would be false positives on the second etc.

Sure, there will probably be some correlation. Depends heavily on how the 3x testing is designed.

T1. Drive up saliva swab by nurse, for serology test.

T2. Re-deploy census peeps to get samples (and census info) from entire neighborhoods.

T3. 23andMe-like home kit express mail to lab

The odds of people getting a FP on all three might not be 1 in 1000, but it probably isn't 1 in 10.

Anyway, as it is, leading 1 in 10 people to falsely believe they are immune, isn't much better than having no test at all.

Assuming false positives aren't correlated even with independently manufactured tests seems like a terrible idea to me, knowing little about medicine.

Sounds like you had a bad experience once, with 'double checking'.

This assumes that the cause for false positive is in the test execution and not a contamination in the sample itself.

This would require that false positives are truly random and not influenced by sample-specific factors. Is this the case? In antibody testing, I would imagine that false positives result from cross-reactivity instead of random chance.

Sure, there are certainly protocols that could be made that would almost ensure FP is due to sample factors.... or even test related factors that would maximize 3x FP (same lab, same lab tech, same machine, same day, back to back, same reagent stock).

We shouldn't adopt those protocols.

That's a correct understanding of specificity.

That would imply the correct rate is closer to 22% than 30%, as of the 64 positives out of 200, we would expect roughly 20 of those to be false positives. (So the remaining 44/200=>22%).

Article says there are 40k residents and 39 deaths.

If 22% of 40k are infected => 8800 infected.

37/8800 implies .42% IFR, broadly consistent with figures elsewhere, especially given that there is a lag time for deaths. (Which will cause the IFR to increase.)

Any bias due to selection etc would basically decrease the denominator on that calculation, increasing the IFR.

(This is all just back of napkin, based on numbers in the article, please check my calculations.)

A 10% false-positive rate is awful. If the actual incidence is 2%, a test with that error rate could overstate the results at about 12%. That combined with the sampling bias could easily result in overstated numbers.

Randomized household sampling would be far preferable. That would obviously take much more time and would expose testers and the household to more risk. But without good methods, research like this and the surveys conducted in Santa Clara and LA counties are potentially worst than useless since they have the potential of misinforming policymakers and the public.

So interestingly, those results from the research paper with more details ( https://www.biomedomics.com/?fldl=3050 and also https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25727 ) credit a different manufacturer called Jiangsu Medomics Medical Technologies. I wonder if that's their subsidiary, or if they're buying the tests from them.

This specificity issue is bothering me.

This 'medical' meaning 'very important' - so if there's a test with 90% accuracy, why on heaven's earth do we not simply run the test 3 times to get 'considerably greater accuracy'?

10% error is so large it's tough to make heads or tails of the data?

Does someone know if this makes sense i.e. if we can simply run the test 3 times and get better data.

Secondly - why are these health authorities not doing proper, state-wide tests?

Here we are with an economy in meltdown, trying to 'model model model' with a 10 Trillion dollar economy wouldn't it make sense for at least ONE (or a few) freaking comprehensive tests that give us some good data?

It has shown to have a 2% false positive rate lower than the 10-15% false positive rate of the general test so its the most accurate test we have.

https://abc7news.com/coronavirus-test-free-testing-update-ac...

If the R0 is truly near six, we have probably climbed near the top of the S curve already all over the place. We will know in a month.

> The 200 participants generally appeared healthy, but about half told the doctors they had had at least one symptom of COVID-19 in the past four weeks.

The total sample size is small and this doesn't seem random enough?

You should be much more worried about bias than about sample size. See also this calculation, where it is discussed that a sample of 400 is more reliable than a slightly biased sample of a million participants:

https://marginalrevolution.com/marginalrevolution/2020/01/bi...

This isn't random at all, lots of people are staying inside and going out to the store once a week or less, they will be dramatically under tested with this methodology.

Meanwhile other people are going out every day, or they know they previously had covid so they consider themselves exempt from shelter in place, or they have a job that requires them to continue heading out to work despite shelter in place (and therefore they have had far more exposure to covid than average).

A third of a random sample taken in Chelsea, Massachusetts:

https://en.wikipedia.org/wiki/Chelsea,_Massachusetts

>Chelsea is [...] directly across the Mystic River from the city of Boston. [...] It is also the second most densely populated city in Massachusetts behind Somerville.

A random sample in a highly affected and extremely dense area is not a model for the whole state.

As soon as I saw this study I Googled Chelsea Mass and in the first few results was an article about how it has been hit hard by the virus. From the article:

> “Chelsea is suffering in this pandemic,” City Manager Tom Ambrosino told WBZ News. “We have, as far as I can determine, the highest infection rate in the Commonwealth.” [1]

With that information the takeaway should be that in the areas with the highest rate of infection we are seeing as much as 30% of the population with antibodies. That's great and an encouraging statistic that seems to indicate the percentage of mild or asymptomatic cases is much higher than originally believed. It does not mean that we are close to herd immunity everywhere. We need randomized testing in many many more communities across the country in order to get the full picture.

[1] https://www.boston.com/news/local-news/2020/04/10/chelsea-ma...

Chelsea is literally the hardest hit town in MA: https://www.nbcboston.com/news/local/the-20-communities-with...

I have some natural doubts over all these antibody tests that indicate herd immunity while covering up major flaws...

Furthermore, a very working class city so I'm sure that a far lower percentage of people are staying home than in many places. [ADDED: Given jobs that don't allow them to stay home.]

And it’s a random sample of people on the street, at some particular place, which may not very representative of the population of Chelsea either.

http://archive.md/UDrho

So halfway to herd immunity?

Go Cows!

It was discussed in another thread that I'll try to dig up, but the percentage of the population required to reach herd immunity goes up the more contagious a virus is. If way more people are infected then we think, then it points to some of the higher R0 estimates being accurate and would therefore require a much higher portion of the population to be immune before we reach herd immunity.

Edit: Here it is: https://news.ycombinator.com/item?id=22819057

My understanding is it's (R0 - 1)/R0

So if R0 is 4 then 3/4 of the population is required to be immune before herd immunity kicks in.

Yeah, I'm not an epidemiologist but I think the principle is simple: it's just a matter of getting the actual transmission rate below 1, so that any outbreak will naturally die out. If in the absence of any immunity the average infected person infects 3 others, but now 2/3 of people are immune, then the average infected person will actually only infect one other. Ditto for a transmission rate of four and 3/4 immunity, and so on.

It's worth noting, though, that the natural tendency during an epidemic is for the total number of infections to exceed the herd immunity threshold; I think the phrase to google is 'herd immunity overshoot'.

(It's also worth noting the serious problems with this study, as pointed out by other commenters.)

> herd immunity overshoot

I saw that too when mucking with simple models. The percent infected overshoots (R0 - 1)/R0. So that is a threshold for herd immunity not the ultimate infection ratio. They're only equivalent under steady state conditions.

Also saw an study released by the CDC that estimated that the initial r0 in Wuhan was 5.8. Explains what happened in Northern Italy and New York. The epidemic achieved break out while most infected were still mildly ill or asymptomatic.

Does this account for the rate of false positives, compounded by the low base rate of disease incidence? When designing such binary estimators, we have a trade off between precision & recall. If the virus affects 1% of the population, and we have a 5% rate of false positives, and close to 100% recall, then ~85% (5:1 odds) of tested positives would be spurious.

It would be nice if it was more widespread than we think. Presumably that would mean we would be closer to the end of the tunnel?

Does anybody how these antibodies are being identified and how distinct they are from other, similar viruses (like the flu)?

Alright. So what's the rate of false positives for that test?

In other words, if this test (and similarly the stanford study in LA) have even a small rate of false positives, combined with any sampling bias, it's going to lead to some misleading conclusions.

Has anyone been able to figure out what the number of COVID deaths are in Chelsea specifically?

The article claims 712 detected cases resulting in (at least) 39 deaths.

Chelsea has a population of about 40k, this isn't mentioned in the article but matches the article's figure of around 1900 cases per 100,000.

Assuming 30% of the population is infected this puts the current number of deaths at around 0.3% of the total infected. There is some bias due to the rate of false positives of the test and the lag in the number of deaths, so the actual fatality rate may well be higher, but the order of magnitude seems consistent with the numbers discussed in previous threads [1].

[1]: https://news.ycombinator.com/item?id=22901311

There have been reports of tests giving up to 30% false positives

This sucks, but also means we’re inching closer to the 70-80% necessary for herd immunity so maybe a silver lining?

This is another irresponsible headline.

I cannot read the article, so correct me if I am wrong, but if what others said is true...

It was not random because they didn't randomly select people, they let people self-select!

If they gave the total number of people who took the test, and total number of people asked, you could at least get a range.

Of course, that range would still be biased to people who are walking around during a time when they should be sheltering in place.

At best, 30% of people who were offered a covid antibody test in public had antibodies.

I think its safe to say that journalists have a responsibility to get the headlines right on this topic or they risk causing a lot of people to make bad decisions after hearing that 30% of people already have this disease. There is a real human cost of this click-bait, and it will be deaths.

This is a strong response from someone who claims not to have read the article.

Is it wrong, though? I did read the article, and I don't think so.

One relevant thing GP may have missed is that the researchers "excluded anyone who had tested positive for the virus in the standard nasal swab test". So they weren't quite trying to directly estimate the proportion of people with antibodies, but the proportion of people without a positive test with antibodies. But I suspect this didn't change the results much: how many people in the area have tested positive and recovered sufficiently to be out in public?

Reply on news.ycombinator.com