The Metric We Need to Manage Covid-19
systrom.com
systrom.com
This study tries to estimate Rt in several European countries from the death rates though there are issues with those statistics too: https://imperialcollegelondon.github.io/covid19estimates/
The data lags behind a bit, but since it measures total number of deaths, it doesn't matter if different countries measure coronoa-related deaths differently. Everyone measures actual deaths. The numbers are hard to fudge for politicians who wants to boost their results.
As a bonus, this also measures how your society is handling the pandemic overall. We know that people will die of the lockdowns as a second-order or third-order effect, and this way we can measure that as well.
Week Monday Observed Expected % Deaths
Deaths Deaths Excess Per 100K
2020-W11 09/03/2020 2,220 2,302 -3.6 19.4
2020-W12 16/03/2020 2,531 2,275 11.3 22.1
2020-W13 23/03/2020 3,116 2,249 38.5 27.3
Side note: Belgium is one of the only countries that includes deaths in carehomes and at home. The deaths in carehomes is almost half of all covid-19 deaths here.
[0] https://epistat.wiv-isp.be/momo/
[1] https://epidemio.wiv-isp.be/ID/Documents/Covid19/COVID-19_Da... (NL or FR only, sorry)
Covid restrictions have also reduced (a lot) some other seasonal diseases like flu and noro/caliciviruses, so you get an effect in the other direction too. When those that die from Covid are in the highest age groups it's not unthinkable that a part of them would have been killed by the flu too, only the flu didn't come this year because of Covid. Perhaps that's why we have a lot Covid deaths but no excess deaths? Who knows. It's too noisy and has too many dependencies to be very useful I think.
To make an absurd example: Duterte in the Philippines has allegedly ordered the military to shoot at people who violate curfew. That helps bring corona-related deaths down drastically, but if your population is dying because you're shooting them or they're starving to death because they can't leave their homes, that's obviously not a good whole-society policy.
And that's why staring at only the corona-related numbers is wrong.
I'm also following the debate in Sweden, which has less restrictions de jure than its neighbours, but people are de facto self-isolating and keeping their distance to almost the same degree. There are lots of people in Sweden screaming that government absolutely must impose a stricter lockdown because the corona-related numbers are worse than in Denmark and Norway, who have stricter lockdowns, therefore Sweden should also follow suit.
But if Sweden doesn't have excess mortality, if Sweden's total mortality rate is the same as Denmark and Norway, while Sweden's economy is less restricted, then I would argue that Sweden is doing the right thing. Or at least not doing the wrong thing. :-)
I disagree with this inference. Different countries have different demography, different weather, different cultural habits, different infrastructures, etc. On top of that, we still don't know how many people would only have benign symptoms when they are infected.
I would refrain anyone to compare pandemic restrictions with a correlation in total death of a country. There are simply too many unknowns, both politically and scientifically
Regardless: when you change pretty much every parameter of society (risk behavior, driving, crime, health care, ...) I don't understand how one can use the overall death statistics and try to attribute them to Covid.
Anecdotally, I definitely find myself trying to be safer right now. I have avoided ladder work on my house and been super careful when using a knife. I just do not want to go to a hospital right now.
I think looking at the overall death statistics are super interesting though. If traffic deaths fall, that is in some ways a byproduct of Covid. If more cancer patients die, that is also a byproduct. All of this goes into factoring how much damage and protection both the disease and the quarantine orders have caused.
Personally I think the best metric is one of the simplest -- the raw percentage of positive tests. This is because it simultaneously captures two important factors: if the percentage is high it means that either you are finding the virus all over the place OR it means your testing capacity is so low that you only are testing highly probably cases. If either of these are true, you should not think about reopening.
https://www.nytimes.com/2020/04/06/well/live/coronavirus-doc...
Many deaths won't be attributed to covid19 but still caused by it.
Here an explanation:
Hong Kong government is also measuring the real time effective reproduction number: https://chp-dashboard.geodata.gov.hk/covid-19/en.html (click on the arrows at the bottom to scroll left and right through various dashboards)
Most obviously that report used the reporting date of deaths as the death date, even though actual death date data was available. The results are dramatically different because of it.
What TFA shows us is the Rt assuming that all cases are known.
The model which knows nothing about testing rate, cannot possibly be used to predict R0 or Rt.
As a simple proof, consider what the model would output I f testing simply stopped. 0 out of 0 tests positive. After several days, the model outputs an Rt=0 with 100% confidence.
If you could account for testing rate, and then attempt to account for missed cases through proxies using hospitalization rates, ILI surveillance data, excess fatality, as well as incubation period, and an asymptomatic rate... maybe then you could draw error bars around an estimate of Rt that would pass the smell test?
If testing was run against 1,000 randomly selected people per geographical area each day, with a positivity rate reported over time, that would provide the perfect basis for making these calculations. The time series of test results that we actually have today require enormous amounts of unpacking before they can be used safely to extrapolate much of anything.
And yes like you say I hope we will see daily sampling soon. As tests are geared up that should be easy enough.
As an aside: The RKI in Germany is collecting weekly samples to track the flu. They have been testing for Corona now for eight weeks. In those tests, they found 11 positive samples out of 1089. I don't know the reliability of the tests though. The numbers are certainly too small to estimate noise after eight weeks.
Of course, it isn’t constant with time, but it is also probably not changing that drastically over short time frames (like the seven days the author uses), so it shouldn’t throw the results off too much.
This of course is a significantly lagging indicator, but it has been long enough that we can see social distancing is working.
What I've looked at so far are IC admissions. As that depends the least on policy I think. Moreover, its the thing we most want to control (i.e. you want peak IC admissions not to go over IC capacity).
I've been using this to judge the effectiveness of measures taken in the Netherlands though. Which is quite different from using it to measure R_0 or R_t in other countries with other data.
Where do you find that data?
I don't think this is true at all. Many people die at home. Many die without having ever been officially tested. I've read that in some places the death toll is only counted from those who were tested and then died in a hospital.
I believe you're confusing fatality and mortality. The correct version of that statement is: "That's why we're talking about mortality, not fatality.
The case fatality rate is the ratio of number of deaths to the number of people who are confirmed to have the disease (which depends a lot on standard of care, reporting, and bureaucratic integrity) whereas mortality rate is the rate of deaths in general.
US testing is so porous as to be meaningless, even today.
Also, I care about bottlenecks like ventilators and ICU staff count, not the 98% of people who don't require hospitalization.
I find all US media and political coverage to completely miss the point, with anecdotes substituting for public health policy. It's like watching a train wreck that won't stop.
We've been testing well over 5000 people per day on average for over a month. Suppose 500 tests per day are randomly administered for the purpose of estimating overall prevalence of the virus. That sample ought to pretty clearly establish the total number of cases within 5%.
I'm sure there are some clever statisticians at the CDC with some fancy Bayesian inference or something who can push the confidence up even higher using all the non-randomly administered tests.
* deaths are also likely to be under reported (although, I agree, probably still better than the total unknown people who haven't been tested)
* a decreasing number of deaths does nothing to fix the actual problem: the possibility of contagion if we all resume normalish activity
My preferred metric has been 'average increase rate over 7 days', in other words, take the new detected cases any given day, calculate what percentage that is over the total for the last week, then average seven of those.
We'll never get a proper idea of how many people are actually infected until we get proper widespread testing, but at least we can get an idea of how bad cases (those that are likely to get tested) are growing. Another data point to look at is how many ICU units are in an area vs. how many ICU COVID cases there are and how many beds are still available. [1] San Francisco has finally started making those stats available:
[1]https://data.sfgov.org/stories/s/San-Francisco-COVID-19-Data...
The one that's used most (the one that appears every day) is here: https://www.arcgis.com/apps/opsdashboard/index.html#/f94c3c9...
That is people who test positive for Covid-19 who then die in hospital.
The other one collected by the office for national statistics is people who die where covid-19 is listed on the death certificate. There's some lag in those numbers.
https://blog.ons.gov.uk/2020/03/31/counting-deaths-involving...
https://www.ons.gov.uk/peoplepopulationandcommunity/healthan...
https://www.ons.gov.uk/peoplepopulationandcommunity/birthsde...
It's hard to know how many deaths occur out of hospital, but it's possible about half of covid-19 deaths are happening in care homes. Covid-19 rips through care homes because the patients are by definition older and more frail; the staff have less PPE and less training in using PPE; the staff (in the UK) are often on very low pay which means they sometimes work in multiple homes; they carehomes sometimes have terrible sick pay terms.
https://twitter.com/AdelinaCoHe/status/1249359297588473858
https://ltccovid.org/2020/04/12/mortality-associated-with-co...
(Lots of caveats, and care needed with these numbers, but)
> Key findings:
> Official data on the numbers of people affected by COVID-19 is not available in many countries
> Due to differences in testing availabilities and policies, and to different approaches to recording deaths, international comparisons are difficult
> Data from 3 epidemiological studies in the United States show that as many as half of people with COVID-19 infections in care homes were asymptomatic (or pre-symptomatic) at the time of testing
> Data from 5 European countries suggest that care home residents have so far accounted for between 42% and 57% of all deaths related to COVID-19.
https://ourworldindata.org/coronavirus#the-growth-rate-of-co...
Infection rate is not the same as number of cases.
That should really be weekly testing of 1000 or more, over a range of backgrounds and wealth. Only then can we really see the impact.
Using mortality is only really useful when we know what the r0 is, or the mortality rate. We don't know either.
I find it difficult to believe with all that testing going on that no one is reserving 3% of the weekly tests to do proper estimation as you suggest.
https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/test...
Bearing in mind that most of that came within the last 4 weeks, its not hard to imagine that it'd be possible to put aside 1000 a week for proper modelling.
After all it would make much more accurate models, which would allow better allocation of resources. [1]https://en.wikipedia.org/wiki/COVID-19_testing
Also this website is crowdsourcing US COVID-19 mortality forecasts if anyone is into data modeling:
https://www.unitarity.com/app/challenges/us-coronavirus-outb...
This website
All we can do now is turn up and down the level of isolation depending on how many ventilators and ICU beds are available. We will have to do that on a sort of feedback, trial and error basis because we lack capacity and what capacity we do have is being wasted.
The only question is whether anyone will be held accountable for this colossal mess and the preventable deaths its caused, or if we will just accept this happening ever 5 or so years...
When the outbreaks in different places are suppressed, you can go back to testing and tracing again.
If active cases can be pushed down to a mange-able level (i.e. a level where contact tracing is a realistic option and outbreaks can be, by and large, contained) a return to relative normalcy could still be possible.
So that’s another one, two months of tight lockdown to push down active cases (to, say, 500 in the NY area). Then it becomes feasible to throw massive amounts of people at those 500 cases and to do aggressive contact tracing and quarantining of all contacts.
Maybe just by people interviewing the infected, calling around, playing detective (a bit slow), maybe also with the help of contact tracing through mobile phones. Also, you obviously would need to be able to test for the virus (ideally) whenever someone shows the slightest hint of related symptoms (and since those are so unspecific and not exactly rare you would have to expand testing capability).
Some measures will obviously stay in place, but many of the most drastic and impactful ones could be reduced since there wouldn’t really be uncontrollable community spread.
That, to me, sounds like a realistic plan (unlike a complete lockdown until then or trying to let the wave wash over us) for the time until we have a vaccine and until that vaccine is availible in sufficient quantities.
Of course, this depends a lot on whether we can push down active cases so low.
South Korea seems to be nearly or already there. Of course we do not know whether China tells the complete truth, but even if they lie quite a bit they are also probably nearly or already there. Outbreaks with exponential growth haven’t happened there at least, of that we can be relatively certain (because that would be hard to hide).
So, yeah, it’s probably possible if we can push effective R a bit below zero for a couple of weeks to get into a position where we can regain control. Maybe. Hopefully.
So are we over the worst of it or is it just starting? Should we increase or decrease our lock down right now?
And before you answer, getting it wrong won't be noticed for at least a week (or more if the 1 week incubation figure is wrong, which it is), and in that time any mistake will kill tens of thousands or burn $100bn dollars.
This is what we get for testing senators and NBA stars when we should have been studying populations. China blind sided us with this once by lying about it and covering it up. But we have blind sided ourselves as well by lacking any leadership and ignoring experts. This whole thing as been a huge lesson in how badly western governments are currently running. The slightest hint of a problem and it's every man for himself.
Testing 1 person makes no difference at all to the treatment of that person because like you say it's not accurate enough to be sure and because we have no (edit: Specific) treatment for Covid-19 that isn't just the same as any other similar infection. Whether you're tested positive, negative or not tested we will ventilate you if you need it and send you home otherwise. So why test any individual person?
But if we had tested (and regularly re-tested) large groups, we would know: * What the range and controlling factors for gestation period are. This is important because you know when and how many people will start arriving in hospitals in the future. * How many people have the disease. Right now, we have very little idea if lots of people have had it already and not noticed and are quarantining for nothing. These are the asymptomatic. * How many people are asymptomatic but infectious. * Whether people can get it twice * What rate people actually require hospital places at. * What the R0 values are AND what they change to with different levels of social isolation.
The lack of accuracy would not really matter because at that scale it just becomes error bars on values and that's fine. We can work with that.
Those are the main pieces of information we need to actually "manage" this disease. Without them we are not managing, we are just guessing. Will their be less infections next week than last week? Will more or less people need ventilators? How many of the nurses and doctors in the building today will be available for work next week? Did closing schools actually help?
Right now, the answer is no one knows. This could all be over in a week when infections suddenly drop because 80% of people have had it. Or it could be that only 3% of people have had it and we will need to maintain all this for another 20 months. No one knows because no one is testing large groups and making big data sets available for analysis. No one doctor can get 100k tests and use them on 20k people over 5 weeks. But the government could. But they're too busy testing each other and giving speeches and shorting the stock market. Sorry if that's a bit tin foil hat.
Q: Is there a reliable test that shows if someone has previously had Covid-19?
Q: If yes, is it in widespread use anywhere?
Any test can be accurate if you use it multiple times. If the coronavirus is 80% accurate, testing everyone twice makes it 96% accurate and three times is 99.2%. Maybe that's the answer.
Even then though, testing individuals is pretty much pointless. It makes no difference to you're treatment.
Not sure if anyone is doing wide testing. The US and UK are not. Germany may be, they seem efficient. Chinas numbers have proven to be total nonsense. Italy seems to have been overwhelmed too quickly to do much.
* in the face of increasing (and volatile) testing capacity, Rt is being significantly over-estimated by looking at case counts. It's improbable CA still had an Rt above 1 the first week of April with covid deaths linear by the second week. More likely Rt was 1 about a week after the SIP (late march).
* Ignoring "herd immunity" effects of Rt. Some of the "under control" states (NY, Louisiana) have had very high infection rates which in their own end has dropped Rt down. NYC is a strong example of this - with > 20% of the city having been infected, well, Rt will drop dramatically just by so many contacts already having immunity.
* Implying this argues for states locking down now. I agree there is heavy evidence favoring lockdowns weeks ago to have avoided many deaths, but at this point, any lockdown is going to take a week to have an impact on even confirmed cases. The most susceptible populations (essential workers) are the very ones exempted from lockdowns (and less susceptible ones have already voluntarily socially distanced), so there's likely not much gain to be had. Sweden is conveniently our low density control group using mostly voluntary measures (WA State is similar through March 23) and its peak passing a week ago is a sign that Rt drops under 1 faster than you think.
EDIT: Turns out there is... https://epiforecasts.io/covid/posts/global/
Why is that? Investigating that seems to be more fruitful avenue.
Perhaps there is more validity to the initial theory that the virus is a lot less virulent in high humidity and heat. Many more observations seem to back that. Low number of cases in India or Brazil etc.
In noisy and unreliable data it's always possible to pick a single data point that supports any desired conclusion. I'd say that all we can conclude is that everybody should stay vigilant.
Reminds me of the Principle of Propaganda - You can prove anything if you ignore enough facts or data.
I think there are effects being seen from the lack of early testing rather than an indication of recent spread during the (very recent) higher temperatures.
Also keep in mind the "excess mortality" [2] - many stories of places where the total death toll has gone up by a lot YoY that the formally reported COVID19 death numbers don't account for.
[1] https://www.huffpost.com/entry/not-just-the-flu-coronavirus-...
[2] https://towardsdatascience.com/covid-19-excess-mortality-fig...
Positive % out of tests: USA ~20%, India: ~5%, Brazil: ~38%.
But at least here in India we test when we are like really really really really sure this person possibly has COVID-19 (if not the very symptoms then either extremely close proximity of a COVID +ve or at a cluster). So that's also there.
Source: I live in New Orleans, returned from Los Angeles March 9.
[1]https://nola.eater.com/2020/3/16/21182099/louisiana-closes-b... [2]https://ny.eater.com/2020/3/15/21180713/restaurant-bar-shutd...
To know you have to investigate that you first need the number that indicates that, no?
This is the same as problem that you see in the White House of America with this son-in-law of president. Systrom has no experience with infectious diseases or study of diseases (or I did not hear about it yet and he does not say about it), and human society needs very much people who are experts right now.
Maybe Systrom could also use tech to help as this is his area of expertise.
Trying to measure Rt seems to be a good idea.
However it's not clear how the proposal calculates Rt. No formulas and no references to raw data. So I am afraid this is not very useful.
This said, I could imagine that an approach which takes the number of confirmed cases and the total number of tests could work to estimate Rt. The idea is that while the number of cases is not a good estimator we can try to get a better estimator using the number of tests as well. It's a bit tricky, however, because a low number of tests could mean that only the probable cases are tested which skews the numbers.