A Review of the Ferguson Imperial Model of Covid-19 Infection
jiggerwit.wordpress.com
jiggerwit.wordpress.com
One possibility is that the many free parameters were used to camouflage the imperfections, like extra barbecue sauce over a burnt kebab.
And while the systematics of population data could well be so great that the Ferguson model is hopelessly unable to offer any insight, one does not prove that by showing the code is unable to produce stupidly implausible distributions (They complain that they can't get a flat population pyramid!).
At the end of the day, the final word is simply this: "A separate question is whether these bugs materially change the predictions of the code. I do not know."
That being said, I am very unsurprised at the results, given what I've seen of scientific code.
However, it's worth noting that the estimates provided by these models in the past have been reasonable: https://statmodeling.stat.columbia.edu/2020/05/08/so-the-rea... (See the P.S. where he realises that the estimates are actually pretty reasonable).
nc =
(p < p0) ? 0
: (p < p0 + p1) ? 1
: 2;The article is diligent in showing the impact of the bugs. The individual impacts seem small, but the fact they found so many instances in one little function is beyond worrying.
For a little quantification, let’s extrapolate the number of such benign bugs (I make it about 1 per 30 LOC), to get about 500 in the whole codebase. Suppose each bug, charitably, has a 1/1000 chance of catastrophically distorting the result. The chance of no such bugs is a mere 60%. For an imho more realistic probability of 1/100, the chance drops to 0.7%.
If the code gives the right answer, it is by pure coincidence.
Most of the COVID projections were off by a lot. The parameters reduced social distancing to a single scalar and often generated those values as outputs rather than genuine inputs.
I looked at various models over the summer knowing that while they may be wrong, they may be useful. But they did not help much at all for understanding trends and relationships. Take for example models that have been modified for mask mandates. During the summer, most of them had trended down to zero with masks “on” but the opposite has since happened.
There is a genuine amount of circular reasoning where models are used to validate social distancing efforts which in turn are incorporated into revised models.
Most of the models I've seen seem to be focused on generating long-term predictions. The big questions people wanted answered at the beginning were "how long will the pandemic last?" and "when will we hit the peak?". And IF you can calibrate your model correctly, for those questions, SIR and its relatives make sense.
But those aren't the questions we should be asking. For public policy decisions, what we really want to know is "is the epidemic spiraling out of control?". For answering that question, a simple exponential fit does well enough in the short term.
Why aren't more models focused on the short-term, when long-term predictions heavily depend on finicky human behavior?
> There is a genuine amount of circular reasoning where models are used to validate social distancing efforts which in turn are incorporated into revised models.
Do you think this reasoning amplifies or reduces social distancing efforts?
Very likely amplifies the effect of social distancing because that is the original bias of researchers. They sought to show that school closures had an effect based on observational studies. Same with other measures. Their statistical modelling attempted to adjust for variables they thought were important.
There were many models that would constantly predict region A to spiral out of control because it's measures were more relaxed than region B. Other assumptions must be wrong but there was very little challenge to the orthodoxy.