Hacker News Reader

Data Robot is definitely cool but you have to be careful you aren't overfitting via hyperparameter tuning.

I think where we'll really start to see the needle move is when we'll have more automated feature engineering. The 'data wrangling' involved here is by far the most expensive part of the process but also best way to improve model performance.

As it turns out, DataRobot does nested cross-validation for model tuning to ensure that the models aren't overfit. We also automate all the out-of-sample/out-of-time validation so you can always check your holdout to be sure.

DataRobot also does some automated feature engineering including text mining, credibility adjustments, transformations of features, ratios and differences of numeric factors, and a bunch of other stuff, too.

The important thing is what they do with it:

At Airbnb, we use machine learning to build customer lifetime value models for guests and hosts. These models allow us to calibrate our marketing spend to expected return at very granular levels, down to the user, if we like.

That's scary.

Automated Machine Learning – A Paradigm Shift Airbnb