Hacker News Reader

Comment by nlh | Hacker News Reader

Sometimes I just absolutely love hacker news more than words can explain.

I am knee deep in a personal project exploring machine learning on tabular data and it’s been consuming my off-hours brain for a while. And I pop open HN on a holiday Monday to find…a package for machine learning on tabular data :)

Curious if anyone has any other suggestions of frameworks or packages to explore. It seems that the state of the art in tabular data got a lot of activity in 2019 and 2020 and the industry’s focus moved on to image processing (DALL-E, Stable Diffusion) so I’m wondering whether there’s been much advancement.

View on HN

If you have tabular data just run with xgboost/lightgbm.

If your data doesn't require preprocessing ... Very often you need to prepare data to use xgboost/lightgbm. You need to fill missing values, convert categoricals/dates/text to numeric, do feature engineering.

What is more, MLJAR AutoML is checking much simpler algorithms for you, like Dummy Models (average response or majority vote), linear models, simple decision trees - because very often you don't need Machine Learning. The xgboost/lightgbm cant do this :)

You do not need to fill missing values or treat categorical with LightGBM.

You can pass the name of the columns that are categoricals when constructing the Booster or Data, and LightGBM will work with them under the hood treating them as a 1 hot encoded.

LightGBM also has a way of automatically treat missing values as either zeros, their own category, or the sample average (I might be mistaken on the last one)

All in all, you still need to do feature engineering and the like, but LightGBM removes a lot of the hassle from Xgboost.

Yeah it seems those are still kings of the hill.

One newbie question maybe you can answer: Can XGboost/LightGBM handle out of band data predictions? The specific regression problem I’m tackling involves price predictions on tabular data (similar the the kaggle housing price problem) and I know classic random forest / decision trees have trouble with time series predictions. Not sure if those models handle that better.

> classic random forest / decision trees have trouble with time series predictions

This is not true. The trick is you have to convert your longitudinal data to be cross-sectional via feature engineering of lagged features. There are also related tricks like expanding datetimes into features like day of week, day of month, etc. This can be a lot of work, though there are software tools which help do this.

Some general time-oriented feature engineering plus a vanilla random forest is a great second baseline (after LOCF), and then if needed you can spend 10x the time tuning a GBM to beat that.

If you have a GPU, checkout fastai's tabular learning stuff; easy feature eng and neural nets with embeddings can do a lot with low effort.

Awesome - thank you! I'll explore further.

Maybe useful:

As part of improving autoML in PyGraphistry, we have been a doing a lot of auto-feature-engineering and auto-clustering work for one-liners like:

g.nodes(pd.read_csv(...)).featurize().umap().plot()

It incorporates a variety of cool base libraries like dirty_cat and transfomers, adds some of our own, and is heavier on text feature column support than most packages here. Think improving upon TopicBERT or dirty_cat for real text & multicolumn data and interactively visualizing, all in one line. We are currently adding end-to-end GPU support for that pipeline (the rest of our stack already supports that), primarily around cuml GPU data frame support.

It powers a lot of our new visual no-code AI layers and is driven by a bunch of enterprise projects in areas like cyber, fraud, social, misinfo, supply chain, & finance :) More niche, this is all feeding into our automatic graph ai (GNN) layers. Graph AI packages can't truly do graphs well until they can do node tables and edge tables well on their own!

https://h2o-release.s3.amazonaws.com/h2o/rel-zumbo/4/index.h...

Reply on news.ycombinator.com