Hacker News Reader

Comment by lr1970 | Hacker News Reader

lr1970

As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasting models that predict the probability distribution of the random variable of interest [0]. Probability distribution is the right language to express prediction values and their uncertainty at the same time. And the decisions based on forecasts oftentimes take the uncertainty measures in consideration, e.g. portfolio risk optimization or buying decisions in e-commerce.

[0] https://en.wikipedia.org/wiki/Probabilistic_forecasting

View on HN

It's worth noting that one of the original authors of Prophet (Sean Taylor) basically agrees with this and is now on a team at Lyft doing probabilistic forecasting for their marketplace [0].

[0] https://twimlai.com/causal-models-in-practice-at-lyft-with-s...

This is a Bayesian model so it of course provides a probability distribution for both the parameters and the forecasted values.

P(model | historic_data) = P(historic_data | model) P(model) / (sum(model) P(historic_data | model) P(model))

Fun fact: if you publish a data point regularly at https://www.microprediction.com/get-predictions then over time, distributional predictions will come to you. It's like a nano-market for live distributional prediction.

This is a good point and true for many things other than timeseries analysis. Any time uncertainty is involved the language of probability is the only thing that makes sense.

And as you say, what matters is generally not the specific values you are uncertain of, but what consequences they can potentially have for your decisions. To know this, you really have to know most values that are possible and how likely they are.

Gaussian processes are great for this. Another easy option is to bootstrap the estimator, though for large models that isn't feasible.

Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.

Prophet with MCMC can produce probabilistic forecasts. But how to choose MCMC priors and measure accuracy of the probabilistic forecast are open questions.

> Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.

Prophet is a GAM (Generalised Additive Model). It decomposes time series in additive components: trend, seasonality, holidays and noise. Most interesting time-series are not so simply decomposable. Making Prophet Bayesian and producing probabilistic forecast by MCMC sampling from trend/seasonality/holiday posteriors still keeps its GAM structure. Might be for a simple exploratory analysis Prophet is a good go-to tool but all the research action is now in Deep Learning Forecasting Models.

Also, IMHO, Prophet deals with individual TS and teaching it to produce vector forecast for multiple TSes at the same time is tricky (or not even possible).

Yes, that’s my understanding too. GAMs have structural limitations that harm flexibility and help interpretability.

One benefit of Bayesian models that they work with relatively little data - and generally provide greater uncertainty in those cases. Do you happen to know of some DL frameworks that behave similarly? I’m eager to learn.

Do you have any favourite libraries for producing such?

https://www.microprediction.com/blog/popular-timeseries-pack...

> Do you have any favourite libraries for producing such?

For modern Deep Learning based probabilistic forecasting you can try DeepAR with parametric likelihood function [0] or Multi-Horizon Quantile RNN (non-parametric) [1]. The implementations of these models in Pytorch and MXnet are scattered all over the place.

[0] https://arxiv.org/abs/1704.04110

[1] https://arxiv.org/abs/1711.11053

EDIT: formatting

And if you want to play with this the gluonts python library makes it very easy. It also includes some other time series models as well! I've been playing with it at work and it's enjoyable.

Reply on news.ycombinator.com