Hacker News Reader

Comment by thearn4 | Hacker News Reader

As interesting as I find the current state of deep learning to be, there is something about random forests that I can't help but find much more cool. Probably the amazing out-of-box performance.

View on HN

You might also be interested in algorithms like Adaboost [0] and its successors, including the various "gradient boosting" algorithms like XGBoost, LightGBM, and the newly-open-sourced Catboost.

0: https://jeremykun.com/2015/05/18/boosting-census/

a great paper on this is Friedman's "gradient boosting machine" paper, where he shows how the boosting idea can be generalised to support a range of different loss functions and underlying approximation schemes (especially trees).

"Greedy function approximation: a gradient boosting machine" - JH Friedman

also the model is analyzable so as to determine the variables which are contributing the most.

You might be interested in this approach to explaining the predictions for any classifier. https://github.com/marcotcr/lime

For some very nice Random Forest visualizations, check out the R package "forestFloor" [0].

I also once started implementing a R package for "partial dependence plots" [1][2], which are popularly associated with Random Forests but aren't specific to them.

[0]: https://CRAN.R-project.org/package=forestFloor

[1]: http://scikit-learn.org/stable/auto_examples/ensemble/plot_p...

[2]: https://github.com/gwerbin/statsplots/blob/master/R/partialp...

Yes, they have very few knobs to turn, which is very attractive.

Reply on news.ycombinator.com