Comment by autokad | Hacker News Reader

Comment by autokad | Hacker News Reader

I did the covid19 week1 and week2 kaggle competitions (I think they had 4 of them) below. If you are interested, this is a fun way to play around with the data, and it shows how hard it is.

This I tried: - Weibull Gamma distributions, but it was impossible to find good parameters for the distributions without exploding. It would only work if I put in an additional parameter saying 99% of the population wouldnt get it. It would come up with good shapes of growth but the predictions were far too below actual values in the future.

- Logistic curves. usually great for countries that already ran up the curve but terrible for ones still in the exponential phase (as the article states). Also kind of useless for countries that didnt even begin their journey up the curve.

- light gbm: good for predicting the next day but terrible many days out. It seems other counties curves do not help that much

- SARIMAX: really good but later predictions would explode, like showing 4 million deaths in france, etc.

I tried to get around these by ensembling them together, but overall I did very poorly at predicting coronvavirus. I still want to get better at this, so if anyone has any good suggestions, please share. Also you can check out what other kagglers have done as well

https://www.kaggle.com/c/covid19-global-forecasting-week-1 https://www.kaggle.com/c/covid19-global-forecasting-week-2

> Weibull, logistic, boosting, state-space

Could you write a model that will tell you how fast your car is going, just by using math? Or are there innumerable latent variables which define a state space so large that the only meaningful prediction you could make is a short-term linear extrapolation?

I mean, what would make anyone think that there is any constant parameter set for a model that would predict COVID-19 case evolution, when governments and the population are taking active measures to combat spread, such as perhaps the most dramatic shift in the physical interconnectivity of human society in history? It's fundamentally impossible to predict something that is changing in an unpredictable manner.

The reason why we can make meaningful inferences about the accuracy of weather models is because we have a good understanding of the underlying physics which decribes the time evolution of the system. However, the combination of the chaotic dynamics of that system and the limitations of simulation yields accurate short-term probablistic predictions and inaccurate long-term probabilistic predictions.

SARIMAX, in contrast, doens't know if it's predicting shampoo sales, $GOOG, or epidemiology. Although I am curious what your exogenous regressors and lag/differencing/seasonal orders were, if you'd care to share. Did you fit them via an information criterion approach?

The problem is just too ill-defined to fit directly with a logistic model or really any general-purpose model (such as gbm, etc). You need better priors, either in the form of good, specified dynamics, or in other terms.

You could do a regularized stratified model [0], for example, in the case where you have plenty of data in one country but not in others, or similar cases which incorporate dynamics (log-log models of growth are also pretty good). Overall, I think it's just too hard of a problem as it is very ill-posed—even if we knew the exact dynamics, any small perturbation in the data samples significantly changes the best fit (in other words, the dynamics of the system are 'chaotic').

-----

[0] https://stanford.edu/~boyd/papers/pdf/eigen_strat.pdf

I briefly studied mathematical biology, and remember there being some debate about whether tumor growth was more accurately modeled by logistic growth or Gompertz growth [1]. I'd be curious to know whether your fits get better or worse if you replace your logistic-based model with a Gompertz-based model.

[1] https://en.wikipedia.org/wiki/Gompertz_function

Thanks, Ill take a look into it. It seems hn has no place for discussion these days and will down-vote anything.

I think you're overthinking things.

The OP makes the basic point that epidemic logistic/S-curves like Covid look like exponential growth for a good portion of their trajectory and so will behave like exponential for a reasonable period in the future. And the data is inherently noisy. Trying to get more than that is counter-productive. Basically, the bulk of the data is pointing to massive death later, the factor stopping that is factors external to the data we have now.

Basically, we know the epidemic is ramping now, we know there's some limit to how far it can go but we don't really know that limit 'till we're close. And trying to get more clever than this results in really bad claims (like guessing we're peaking and abandoning caution).

I'd also add testing results have been unreliable based on real world limits to testing (limits to number of tests, general screw-ups). Death data is more reliable than testing data but delayed by infections taking time to kill people.

The "quick and dirty" modeling of Tomas Pueyo has been more successful than clever actions.

See: https://medium.com/@tomaspueyo/coronavirus-act-today-or-peop...

In contrast, https://covid-19.direct has good data but too clever algorithm for spotting peaks. If you browse their data, you get used to garbage "forecasts" of where the peak is.

https://medium.com/@tomaspueyo/coronavirus-act-today-or-peop...

Reply on news.ycombinator.com