Hacker News Reader

Comment by xamuel | Hacker News Reader

makomk posted an interesting comment on another thread here 12 hours ago, which I'll paste below. (from https://news.ycombinator.com/item?id=23233097)

I think I've got a good guess what data she didn't want removed or changed, and it's not exactly good for her case. On May the 5th, they day she was removed from her position, there was an all-time high spike of 113 reported coronavirus deaths due to Florida reporting a bunch of old deaths all at once: https://www.tampabay.com/news/health/2020/05/15/florida-adds...

This did not represent an actual spike in deaths which had in reality been declining, but if I remember rightly some people did spin it as deaths increasing due to the lockdown being lifted. Deaths by date reported are conspicuously missing from the current version of the dashboard; it only has dates by date of death, which is far less favourable to the claim that lifting the lockdown is causing an increase in deaths.

View on HN

Let me enumerate the doubts I have about makomk's speculative guess.

1. The linked article in the comment reflects a clear understanding of the lag between occurrence and report dates, which points to the dashboard not having been in any kind of misleading state.

2. Even when a lot of politics is being played, it does not seem plausible that people were spinning deaths reported May 5 as being due to the May 4 reopening.

3. The "in reality had been declining" part fails to acknowledge the way reporting by occurrence date always makes graphs have a decline at the end (see rovolo's comment, sibling to mine)

Even if the data was misreported, which is a possibility, censoring or deleting it in some way is the worst option. Lets say Florid's growth for that month was 30%. But because of bad reporting it thought it had a growth rate of 15%, until the end of the month when all of the missed cases were reported. That spike will not be accurate, but just deleting it would make it look like Florida still had a 15% growth rate. Reviewing the full months data to correct those issues would give the clearest report, but reporting the raw data is more honest than just deleting the anomaly.

The trade off is between measurement bias and selection bias. Typically you want to be dealing with measurement bias and not selection bias, since the latter is often impossible to account for.

You are correct that it is preferable to report the raw data, but it is more than just "honest"! Even with that data `incorrectly` on the wrong day, when binned as a full month the growth rate will be correct. That is, the data is now more correct than if the late reported deaths were not included.

I believe this is effectively the same as Intention to Treat Analysis (https://en.wikipedia.org/wiki/Intention-to-treat_analysis).

> "it does not seem plausible that people were spinning deaths reported May 5 as being due to the May 4 reopening."

You've got to be kidding. Here on Hacker News people were buying the very same sort of spin from an article about the 1918 flu epidemic and San Francisco (I think that was the city). I've seen lots of people on either "side" of this issue buy into absolutely outrageous claims that are clearly just spin.

For example, weeks ago somebody on the "open up now" side graphed the data from the CDC showing that deaths around the country weren't increasing, but in fact plummeting. The page with the data is explicit in multiple places that the data is incomplete for many weeks into the past. Then, weeks later I heard that the official death toll from Covid-19 had been cut by 50%. When I asked for a source, it was that same page with data that lags by up to 8 weeks.

Another example: somebody on the "panic now" side reported (in the news, IIRC) that Michigan's positive test ratio was skyrocketing and had reached 47%. The only table that contained positive vs negative test results clearly had not been reporting the negative tests at all. They should have just gone ahead and claimed that 100% of the tests in MI were positive, since that was what the table showed for the previous 5 days.

The fake news around Covid-19 has been driving me absolutely crazy on both sides.

Framing two sides as "open up now" and "panic now" does not help your case.

I think putting both viewpoints down basically implied he was impartial and and talking of those who had an agenda.

It helps reduce verbiage.

Calling “stay in lockdown” a panicked response seems like picking a side to me.

But I didn’t call it that. I called lying about what was happening in MI in order to make people think things were way worse than they really are, the “panic now” side.

Do you not agree that such lies are bad? Perhaps you think the end justifies the means, but in that case, you might want to consider what telling lies does to the public’s perception of your trustworthiness when you tell them it’s necessary to stay in lockdown.

And then there’s all the real reasons not to panic. Because hoarding is bad for everybody. Because failing to seek necessary medical care for your ruptured appendix because you’re afraid of getting Covid-19 as a 28-year old would be pretty bad. And on and on.

What possible good do the lies I described above accomplish?

You make a good point. I'll revise my statement:

It does not seem plausible that reasonable people would suggest suppressing data just because some people tried to spin deaths reported May 5 as being due to the May 4 reopening.

If information sources in either of the examples you gave were later suppressed due to the extreme spin put on them, please let me know.

For the curious: Michigan's positive test rate on 19May2020 was: 435 new cases/12726 tests = 0.034, or 3.4%. Which is pretty good and means they are finding most cases within the section of the community they are testing.

For comparison, Australian testing rates (some of the best in the world) are around 0.5% to 1% in the week after the peak of cases.

Why does everything in the US always have to be about two strictly opposing sides?

First there is the assumption that you have to "pick a team", and that every voiced opinion also has done so.

Then there is the assumption that the voices from the far extremes of these sides are the ones worth listening to.

https://xkcd.com/2305/

The fact that most people agree (or did at some point) about how we should respond to Covid-19 has no bearing on my comment that I can figure out.

Using date-of-death based statistics does indeed cause a misleading drop-off in the last few days - there's no perfect way of reporting uncertain information like this in real time - and I should probably have noted that, yeah, but the decline was longer term than that. Though in any case, it looks like my guess was wrong and the problem with Florida stats was even more embarrassing than that. Like, I'd assume if the supposed data manipulation had to do with drumming up support for reopening it'd have some relevance to what was happening now...

Massachusetts (where I live) just retroactively restated their case and death data by test/death date vs. reporting date so it more closely aligns to when a person was experiencing sickness or death. I find it far more useful now. The curve is now smoother, and you can see just how lumpy reporting timing was making daily releases of numbers. It has the knock on effect of making the numbers for the last seven or so days liable to change every day, but it’s still a better picture of what’s going on. This should be the norm.

Colorado includes all of this information. It is useful to see it in different ways. But I have definitely found the deaths-by-date-of-death to be the most useful (it is also the most encouraging).

> I have definitely found the deaths-by-date-of-death to be the most useful

Tracking deaths is actually the worst, most lagging indicator. It takes a very long time to die.

The most useful was the CDC tracking cases-by-illness-onset, that gave you a great understanding of what the trend line really looked like over the last couple of days.

All indicators have pros and cons. Deaths by date of death is the most lagging indicator, which is bad for in-the-moment decision making, but it is also the most direct measure of the thing we should care most about, minimizing death-causing infections. So you can look at a graph of deaths by date, and if it is way down (like it is in Colorado right now), then you know that things were going well over the last few weeks. This doesn't give us any data on how well we're doing now, during our partial re-opening, but it will give us that information over the next few weeks, in a way that we can compare with what we were doing over the last few weeks.

Cases by onset date is pretty good too, but there is this period of some unknown number of days where it is misleading because it is too incomplete. The Colorado tracker shades ten days as "potentially unreported cases", but I always wonder how they chose ten and whether that's conservative or aggressive. The death by date of death has the benefit of not being incomplete in that way. Another problem I see with cases by onset date is that I don't really care that much about how many cases there are, not directly. If there were a huge number of cases, but no deaths, that would be a bummer because being sick sucks, but it would not justify our extreme reaction. What we care about are the deaths. So I like direct measures of that.

Cases by reported date doesn't seem very useful at all, for anything. Percent positive tests by day is a similar measure that seems more useful.

But I'm an amateur, this is just what I think is and isn't useful for reasoning about what's going on.

That's called an 'epi curve' and it's a very common thing, but the data usually lags somewhat.

'Date of reporting of case' and 'date of onset of symptoms' are two different things, both useful.

The epi curve has the problem that it always looks like cases are declining because the most recent few days are 'not filled in yet' by cases that have been reported.

I don't think it's anti-scientific to report either way, so long as the dats is well represented.

If you have been following daily reported death "headlines" for MA, it doesn't line up with the state's per-day graphs; see page 9 [0]. I assume they're investigating them and going back and updating the daily totals as the graph seems to lag a few days, but it's not clear to me what methodology they are using. They have been doing this for a while.

[0] https://www.mass.gov/doc/covid-19-dashboard-may-19-2020/down...

Please don't copy-paste comments on HN, and especially not someone else's. Copying lowers signal/noise and makes thread merging a nightmare, since we end up with duplicates.

If you want to reference what someone else (or you yourself) already posted, please do so with a link. Adding info about why you're linking to it is also welcome.

Edit: Actually it would be fine to quote from what you're linking to as well. This did lead to good discussion and I don't mean to interfere with that!

If you want to see what messy reporting looks like just look at France data: https://www.worldometers.info/coronavirus/country/france/

It is transparent, and you can find in official sources what is counted and what is not. But just looking at the global statistics, it is hard to know what's going on. We even have people "resurrecting".

The biggest reason for all this mess is that we combine data sources with different levels of reliability.

- Many things go unreported on week-ends and holidays. So we often see artificial spikes on Mondays, and Tuesdays.

- Hospital deaths are reliable but incomplete, and taken by themselves, with week-ends smoothed out, you see a rather smooth curve that makes sense.

- EMS/EHPAD (nursing homes) deaths are less reliable. Patients dying while showing symptoms are considered COVID-19 victims, even if not tested. It can be reverted at a later date. Reporting can take more time, and in fact wasn't considered at all during early statistics.

- People dying at home are not counted at all, too difficult to keep track of. We do have global mortality stats for all causes though.

- During the peak of the pandemic the testing policy was to test only if necessary for the health of the patient or medical personnel, not for statistical reasons. It means a lot of underreporting.

> We even have people "resurrecting".

That's a common problem. I spent a while contracting with an insurance company a decade or so ago, and one of my fellow developers was tasked with coding an "un-decease person" feature. It was non-trivial.

I think that using date-of-report is better for lay-people. Pro/cons:

- There's a one time spike when the methodology changes

= The data lags reality on the ground (true for both date-of-report and date-of-death)

+ Past dates aren't updated, so there isn't a continuous bias to show a slowdown in the number of new deaths

I think that the last point is the most important because it's super easy for people to misinterpret provisional data. People were talking about covid-19 not causing excess mortality because using date-of-death always shows a dip for the present: https://twitter.com/TylerMorganMe/status/1248452375162925057

You can also see the same thing in the top comment on this Euromomo thread: https://news.ycombinator.com/item?id=22866238

> Another very surprising thing is to see no excess mortality in Sweden, a country criticized for a laxer strategy against COVID-19. I was not expecting to see that.

That was week 14 for https://euromomo.eu/graphs-and-maps/ . In fact, week 14's excess death z-score for Sweden was in the "very high excess" category after the data was collected.

There is a third option here. Don't publish data until it's collected. This means you don't publish data for deaths on a date until all the deaths on that date have been reported. Then you don't have to retroactively update anything because you didn't publish numbers before they're finalized.

This option is clearly better than publishing date of report as date of death, because doing that is inaccurate. The data is really from events a week or more ago when you think it's from today, and you can get clusters like this when many reports come in at once.

The disadvantage is that you then have newer data that isn't being published because it isn't completely in yet. But if you are going to publish incomplete data then it should still be date of death and the dates without full reporting should be annotated so that people understand that they're looking at incomplete data -- which they are, so it's very important for that to be understood.

It's a sliding scale between immediacy and accuracy, with a very high upper length of time for perfect accuracy. I work with death record statistics for a state government, and our reports use data from 2 years ago. A big reason is gathering reports for residents who died in other states.

I know the CDC has a good system for sharing infectious disease diagnoses (https://wwwn.cdc.gov/nndss). I don't know if there's a similar rapid-collection system for death records. We usually don't need to track deaths with this urgency.

A possible compromise is to model a delayed reporting factor. Not sure if there could be a good enough model, though.

> don't publish data for deaths on a date until all the deaths on that date have been reported

Based on the animation I linked, it takes around 15 weeks for an apparent decrease in pneumonia deaths to rise to a normal number. 15 weeks ago was Feb 4. (Granted, pneumonia death data collection in normal times will be different than covid-specific data now). The numbers do get more accurate with time though. There is a trade-off between accuracy and timeliness.

Lay-people don’t really dig into the data and don’t compare i dates for changing stats. Tracking deaths by date of occurrence is much more useful than date of report, even if it means that you have to restate figures sometimes. We see that happen a lot with economic figures where they restart the previous month or quarter’s stats.

Restating the figures isn't necessarily bad. The issue with mortality revisions is that they're always revised upward; whereas the economic revisions can go either way.

Using date-of-death introduces a consistent bias where recent deaths are undercounted compared to earlier deaths. People care about whether deaths are currently slowing or increasing. Date-of-report may be inaccurate, but it doesn't bias the trajectory of deaths consistently in either direction.

Please don't cite euromomo as a source:

* They stopped graphing the per country raw numbers, which makes it impossible to put things in perspective: what is the comparative excess mortality per million people?

* They use deceptive graphs that do not start yaxis at zero

* They refuse to publish the raw dataset for independent processing.

I wasn't using euromomo as a source, I was using a thread on euromomo as an example of faulty interpretations of provisional date-of-death data.

"One day before a top Florida Department of Health data manager lost her role maintaining the state’s COVID-19 data, she objected to the removal of records showing people had symptoms or positive tests before the cases were announced, according to internal emails obtained by the Tampa Bay Times."

https://www.tampabay.com/news/health/2020/05/19/florida-heal...

So I guess "makomk" was wrong. And your admiration for his baseless speculation was also wrong.

I wonder what percentage of all HN posts, almost none of which receive such honours as being reposted by an admirer, manage this sort of trifecta of missing its prediction, never actually having a meaningful argument or mechanism for the prediction, and, by its own admission, smearing the person concerned with these unsubstantiated claims of lying and/or at least acting in bad faith.

lockdown has had no effect on cases/deaths. just look at the data in germany or other countries which have lifted lockdown since 4 weeks. no difference.

Germany didn't.

It is still illegal to invite 2 friends at the same time, for example.

Also, wearing face masks inside shops and in public is still mandatory.

Yes, the lockdown is gradually being weakened, but far from being over.

Reply on news.ycombinator.com