Darts: A Python library for easy manipulation and forecasting of time series | Hacker News Reader

Darts: A Python library for easy manipulation and forecasting of time series

github.com

239 pointsgrzracz43 comments

Hi! I'm one core developer (and creator) of the library. Thanks for all the comments. I just wanted to highlight a couple of things that we think are quite cool about Darts:

* It makes using all sorts of forecasting models (from ARIMA to deep learning) easy, using fit() and predict(), similar to scikit-learn.

* It's easy to fit deep learning and other ML-based models on multiple time series, potentially on big datasets too. The time series can be multivariate.

* Darts is not only wrapping existing models. We also have our own implementations, for instance of TCN (Temporal Convolutional Networks), or adaptations N-BEATS (which we extended to handle multivariate series), DeepAR and others.

* Darts makes it very easy to include past and/or future covariates as inputs for the predictions.

* Some models offer probabilistic forecasts; sometimes with the possibility to configure your favourite likelihood function (e.g. Gaussian for continuous values or Poisson for discrete values).

* Everything uses the "TimeSeries" class, which makes the API consistent across tools and models, and make it harder to make mistakes. For instance it's easy to consume the output of one model by another model, and all models can be backtested the same way.

I love to see more time series models becoming available in an easy-to-use format. There's always been such a gap between what is possible and what is convenient to use, much moreso than with other kinds of models.

This was also one of the areas where R always had better options than Python, but that seems to be gradually changing as well.

Darts looks very thorough and user-friendly, it makes me really want to work on a forecasting project!

It might be very helpful to readers/users if you could add a section to your documentation comparing Darts to Tslearn [0] (edit, and Sktime [1]), which already has a lot of time series models with the Scikit-learn style interface.

It would also be helpful to have some kind of writeup that explains the TimeSeries data structure and why you use that, instead of just a Series/DataFrame.

Finally - you really shouldn't say "non-Facebook alternative", because your Prophet implementation is literally a wrapper around Facebook's Prophet library. If anything, I suggest moving the Prophet, Torch, and Pmdarima dependencies to setuptools "extras", so you don't force the users to depend on those projects.

[0]: https://tslearn.readthedocs.io/en/stable/index.html

[1]: https://news.ycombinator.com/item?id=28155384

Thanks for the feedback, I absolutely agree about the need for easy-to-use tools for dealing with time series. This is exactly the motivation that prompted us to work on Darts initially.

I like your suggestions of adding comparison to the few other libraries out there, as well as explaining the need for having our own TimeSeries data structure. We should try to do that sometime soon.

Concerning dependencies, we already have some dependencies as extras. "pip install darts" will install everything, but "pip install u8darts" will install only the core (without Prophet and pmdarima), or "pip install u8darts[torch]" only the core+pytorch models.

Regarding deps, I just saw that you have two setup.py files, and one of them has the `extras` listed. Good stuff!

> I suggest moving the Prophet, Torch, and Pmdarima dependencies to setuptools "extras",

Yep, the dependence on Prophet prevented me to quickly try darts on a Windows machine because of the pystan issue with prophet:

https://github.com/facebook/prophet/issues/732

May try it later on Linux.

You can try installing it using

    pip install 'u8darts[torch]'

This will give you everything besides Prophet and pmdarima.

Will you add support for anomaly detection or classification (something like predictive maintenance)?

In short: yes, at least anomaly detection. That's one of the main things on our to-do list.

Do you have any plans to implement some sort of model averaging or stacking? I believe it would bring great benefits to this landscape to have a working implementation of hierarchical stacking across various backends wrapped in a Python library.

We do have ensemble models. Something like

    model = NaiveEnsembleModel([model1, model2, ...])
    model.fit(my_series)
    prediction = model.predict()

Will return an average prediction. Look at RegressionEnsembleModel for an ensemble model which uses a regression model to learn how to combine the individual forecasts.

At the moment Darts doesn't have hierarchical reconciliation methods (if that's what you meant), but it's on the backlog :)

FYI, Hacker News has code formatting if you indent by 4 spaces:

    model = NaiveEnsembleModel([model1, model2, ...])
    model.fit(my_series)
    prediction = model.predict()

Oh, nice, I didn't know that. I've edited my reply, thanks!

Hey guys,

If you are interested in a unified interface for time series learning tasks, have a look on sktime:

https://github.com/alan-turing-institute/sktime

https://www.sktime.org/en/stable/

It provides specialized time series algorithms and scikit-learn compatible tools to build, tune and validate time series models for multiple learning problems.

We have been working hard on this open-source project for a long time and would be glad to hear your opinion.

I have been waiting for the Time Series Regression user guide section.

The project sounds really cool but I will for sure get into it more once that guide is available.

Darts is mostly a wrapper for a bunch of other timeseries forecasting libraries, and provides a single interface to work with them. It's not really a Facebook alternative. Facebook's Prophet library is one of the forecasting libraries used by Darts.

In some cases Darts is wrapping around existing models (like Prophet, or statsmodels-based models for instance); in other cases we wrote our own implementations, so it's really a mix.

I didn't realize that. Darts is a great library that we use all the time. Thanks for all the hard work!

You're welcome, glad you like it and thanks for the feedback :)

I was trying to use Darts earlier for some multivariate data and was struggling to figure out how to use it for it and eventually just gave up and switched to making my own code.

Is there a good "how to" multivariate data example? Or is it just turning every column in my pandas dataframe into a series to pass into the covariates array?

And rather than just bother you, is there a discord/forum to ask questions on darts?

> Or is it just turning every column in my pandas dataframe into a series to pass into the covariates array?

Basically if you have a multivariate series represented as a pandas dataframe with several columns, the way to go is to create your TimeSeries by calling TimeSeries.from_dataframe(my_df). That will return a multivariate time series.

We don't yet have a discord channel, but I'm planning to open a Slack channel sometime soon. If you have other questions feel free to drop me an email: julien@unit8.co

Alternative to Facebook Kats, not Facebook Prophet

That makes more sense. I'll have to look into Kats as an alternative to darts :)

Yeah, exactly, I am now excited about Kats so actually the title was useful for me at least.

I am writing code for a time series NN at the moment so am keen to check out darts

conda install -c conda-forge -c pytorch pip prophet pytorch cpuonly

pip install darts

Why not install darts with conda as well?

I am getting some errors runnning the on a debian system install

when running the example script I get this: Importing plotly failed. Interactive plots will not work. /home/peter/anaconda3/envs/darts/lib/python3.7/site-packages/statsmodels/tsa/holtwinters/model.py:429: FutureWarning: After 0.13 initialization must be handled at model creation FutureWarning,

I guess its a library compatabilty but feedback is important.

You can install darts with conda using

    conda install u8darts-all

We are working on improving our support of conda based installs.

On a side note: I love the example with the airtravel time series (# of passengers). It fits almost perfectly but I love to see how it holds up with data from around March 2020 until now ;-).

Disclaimer: studied econometrics so I will try do this on my own :D

How does this compare with https://github.com/linkedin/greykite?

I would say that compared to Greykite, Darts really attempts to unify a wide variety of forecasting models under a common simple and user-friendly API. There are many differences, but for instance, AFAIK there's no deep learning model in Greykite (it focuses on two algorithms: their built-in algorithm and Prophet), whereas Darts tries to lower the barrier for using deep learning models for forecasting. Crucially for ML-based models, it also means being able to train on multiple (possibly thousands or more) of possibly multi-dimensional time series.

Can't wait for the Dart port.

Sigh, downvoted again. HN, the place where humor goes to die. I mean, sure, not the best pun ever but is it really necessary to downvote, I wonder?!

> Darts: Non-Facebook alternative for timeseries forecasting

The title of the post seems very editorialised.

First of all, being non-Facebook is hardly meaningful when we talk about open source tools. Secondly, the project doesn't advertise itself as being non-Facebook, the poster has added this. And lastly, it's false - from the prerequisites in the readme:

> Some of the models depend on prophet and torch

Yes, that's offtopic and baity. We've changed the title now. (Submitted title was "Darts: Non-Facebook alternative for timeseries forecasting")

Submitters: the site guidelines ask you to rewrite titles only to make them less baity. Less is not more! Not in this context at least.

"Please use the original title, unless it is misleading or linkbait; don't editorialize."

https://news.ycombinator.com/newsguidelines.html

Using Prophet is optional :) To be an alternative doesn't mean to cast everything useful aside

Edit: it's important to note that this is an alternative to Facebook Kats, I should've probably added this in the title and that's my bad

What do you mean by "optional"?. You are including it [2] [3] [4] as one of the options (not re-implementation of the original paper [1]) and it is one of the dependencies [4] (Prophet [5] is an open-source library built by facebook research).

I think we should distinguish between science/open-source and policies when we mention open-source projects.

*References*: [1]: https://facebook.github.io/prophet/#:~:text=Prophet%20is%20a....

[2]: https://github.com/unit8co/darts/blob/master/darts/models/pr...

[3]: https://github.com/unit8co/darts/blob/master/setup.py#L12

[4]: https://github.com/unit8co/darts/blob/master/requirements/pr...

[5]: https://github.com/facebook/prophet

If I put out a buffet where some items are vegetarian and some are not, I don't advertise it as a vegetarian buffet with the excuse that the nonvegetarian foods are optional.

That’s actually exactly how vegetarian buffets work.

If you want to avoid Facebook with this library you can. That may not be the case with every library so folks who care about such things appreciate the callout. If you don’t care you don’t have to.

What’s the problem?

> That’s actually exactly how vegetarian buffets work.

I'm going to go out on a limb and guess you don't visit many restaurants that advertise vegetarian buffets

As a side note when in Brazil I felt like "vegetarian" food meant "just little bit of meat" and "caipirinha without sugar" meant "just don't mix (the layer of sugar at the bottom)" :) Did love it though.

Lol people will roast you if you say vegetarian buffet and serve meat there.

As an apropos there’s a bomb vegan Korean buffet in Dallas.

> That’s actually exactly how vegetarian buffets work.

That’s just a buffet. When you use a qualifier like non/vegetarian, then the assumption is that the buffet would comprise of that.

> it's important to note that this is an alternative to Facebook Kats

TIL about Kats. My assumption was that this was a reference to Prophet, which has not been treated favorably lately on HN.

What has Facebook got to do with time-series forecasting?? Prophet?? Real time series work means rolling your own tools and understanding exactly how they function ;-). In C preferably, while flogging yourself at the same time. Just kidding.. Sort of.

I opened this page because of the friendly "Non-Facebook alternative...". In my view, we should ignore software by companies that actively seek to destroy our societies.

Reply on news.ycombinator.com