Hacker News Reader

Comment by b_tterc_p | Hacker News Reader

While there’s nothing wrong with random forests, they’re a bit of a red flag for me as they’re easy to implement without any real understanding of what’s going on. A lot of junior data scientists just default to saying random forest to solve any problem because it tends to have the most predictive power of the models they’re comfortable with. That’s a bad sign.

View on HN

I am probably one of the junior DS you are referring to. But, I genuinely want to know the reason of using anything other than gradient boost tree to do classification on structured data.

It depends on your goal, and the nature of the problem. If you need to explain something in simplest terms, maybe use regularized logistic regression. If you need to make sensitive decisions, maybe a tree would be best because you have a clear sense of the variance in your answers at each node.

There’s nothing wrong with random forest. It’s a perfectly good model. But when it is someone’s only tool, it implies they both don’t know much much about the toolkit, and also how that one particular tool works.

I rarely use anything but linear models, trees and forests fwiw.

Is there a place that tells you: If you have this type of data and want this kind of answer, here's the best algorithm (and why)??

https://scikit-learn.org/stable/tutorial/machine_learning_ma...

What's the bad sign there? If juniors are reaching for RF as a first-pass over say, (logistic) regression, that seems like a _great_ sign; no other approach has such a high average performance-effort ratio. For non-huge, naturally tabular problems, it usually takes me 10x or more the effort to beat the very first RF I train. Developer time matters!

If they start going to GBM's or neural nets first...I'd call _that_ a bad sign (and it happens).

I can understand why a neural networks first is bad, but I don't understand the problem with GBM's

On the other hand, if it works it works.

There are also lots of good ways to peer into the inner workings of a tree ensemble model nowadays. It's not laid out plainly for you like a linear model, but it's not an impenetrable black box as people like to suggest.

I completely agree. Tools like shap are really useful for peering into tree-based models.

https://github.com/slundberg/shap

That’s true, but they also tend not to know them. Or worse, they point to variable importance and don’t know how it works which leads them to the wrong conclusion.

The real differentiation will come from feature engineering. Random forests and boosted trees (the other fire-and-forget model of choice) can tell you a lot about the data set from their validation performance under tuning. It might be that you've got a straightforward situation, but often the next step is to dig into what the model is doing and see if you can better optimize the features.

Reply on news.ycombinator.com