I've always wanted my own gradient boosting machine implementation to compete with Xgboost and LightGBM. There's a huge space of tricks around regularization, randomization, tree structure, etc., that few people are exploring because neural nets are exploding.

So far I've roughly caught up in speed and accuracy with a few original tricks (and 1/20 of the features), but no real breakthroughs.