Hacker News Reader

hamid9on LinearBoost: Faster and more accurate than XGBoost and LightGBM on 7 famous data

The latest version of LinearBoost classifier is released! https://github.com/LinearBoost/linearboost-classifier In benchmarks on 7 well-known datasets (Breast Cancer Wisconsin, Heart Disease, Pima Indians Diabetes Database, Banknote Authentication, Haberman's Survival, Loan Status Prediction, and PCMAC), LinearBoost achieved these results:

    It outperformed XGBoost on F1 score on all of the seven datasets
    It outperformed LightGBM on F1 score on five of seven datasets
    It reduced the runtime by up to 98% compared to XGBoost and LightGBM
    It achieved competitive F1 scores with CatBoost, while being much faster
    LinearBoost is a customized boosted version of SEFR, a super-fast linear classifier. It considers all of the features simultaneously instead of picking them one by one (as in Decision Trees), and so makes a more robust decision making at each step.
    This is a side project, and authors work on it in their spare time. However, it can be a starting point to utilize linear classifiers in boosting to get efficiency and accuracy.

hamid9on Our classifier outperforms CatBoost, XGBoost, LightGBM on 5 benchmark datasets

Thank you for bringing up this issue! Our plan is to release the paper in a month, but let's see how it goes. Feel free to reach out to me if you have any questions!

hamid9on Our classifier outperforms CatBoost, XGBoost, LightGBM on 5 benchmark datasets

Thank you for your message and pointing it out! I think it needs some clarification (I will update the documentations as well). The classification algorithm that you mentioned is SEFR, which is energy-efficient, but not as accurate as other algorithms. LinearBoost is the boosted version of SEFR, and it has superior F1 in 5 benchmark datasets over GBDTs. So, SEFR to LinearBoost is somehow like Decision Tree to CatBoost. SEFR is fast, and by boosting SEFR, we have LinearBoost which is slower but accurate. The results will be provided as a paper, but now, they are in the GitHub Repository's README file.

hamid9on Our classifier outperforms CatBoost, XGBoost, LightGBM on 5 benchmark datasets

Hi All!

We're happy to share LinearBoost, our latest development in machine learning classification algorithms. LinearBoost is based on boosting a linear classifier to significantly enhance performance. Our testing shows it outperforms traditional GBDT algorithms in terms of accuracy and response time across five well-known datasets. The key to LinearBoost's enhanced performance lies in its approach at each estimator stage. Unlike decision trees used in GBDTs, which select features sequentially, LinearBoost utilizes a linear classifier as its building block, considering all available features simultaneously. This comprehensive feature integration allows for more robust decision-making processes at every step.

We believe LinearBoost can be a valuable tool for both academic research and real-world applications. Check out our results and code in our GitHub repo: https://github.com/LinearBoost/linearboost-classifier

We'd love to get your feedback and suggestions for further improvements!