Comment by dmichulke | Hacker News Reader

Comment by dmichulke | Hacker News Reader

I have been working in the same field, quite a few years in my own forecasting company and then later on as a side project.

Not wanting to steal the thunder of Deepmind, but I feel should mention my site which is currently beta as in "works for a few electricity power load and supply forecasting scenarios" and quite well so. Also, it's out there, cheap and fast.

https://ausblick.cryptoport.net

Unfortunately, it's only in German but the basic rule is:

- upload an excel (or csv) file

- wait a minute or so (for a typical dataset of say 3 years of hourly data)

- get back the results

There are a few design rules how the excel file should look like, e.g.,

- a date/time series per sheet (it accepts most English and European formats) and as many number series as you want (always use a decimal point instead of comma here)

- one series has to have consecutive missing numbers at the end, this will be the target value

Other than that, it should just work.

Feel free to comment.

PS: The examples might be the best explanation - https://ausblick.cryptoport.net/examples

What prediction algorithm are you using for this? Not sure if it's mentioned elsewhere on the site

I'd also be interested in hearing a bit more about this.

I have previously built such a tool for retail sales forecasting (based on previous sales). The stack used R + Rshiny (web app). Method FB's `prophet` as it seemed faster and more accounting of holidays and other variations.

No, it's not mentioned. It's currently a mix of very simple techniques

- a number of smart features (usually a few k) depending on the series (using lags, aggregates, curve fits, combinations of features, ...)

- an iterative algorithm that selects features using maximum relevance (~ correlation with the target) / minimum redundancy and adds them to the model

- simple pca and ridge regression (because it's fast)

- a few optimizations of the final model (removing features, selecting a better ridge regression alpha with CV, ...)

The stack is pure Clojure / Clojurescript.

How did you find developing your models in clojure? I would love to switch away from python for ml but it seems to just have the ecosystem.

Actually, I use both but my production models are in Clojure.

I often end up implementing minor things myself using lower level abstractions (e.g., Linear Regressions or PCA with whitening using Matrix libraries) and I check the results and/or try new things using scikit-learn.

So in general, I'd say I do the programming (outputing intermediate CSVs, tests, web service, thread handling, UI, ...) in Clojure(Script), and try other approaches (e.g., other models/parameters/...) in Python.

I'm quite happy with this pipeline but probably to some extent because I really love to understand how things work and nothing pushes you to learn as much as a missing function in your ML library :-)

Yes, me as well.

Reply on news.ycombinator.com