Anyone has more insight into why? if I try to grid search hyperparameters for a random forest takes ages in a single machine with scikit-learn. I only ever see GPUs in neural networks. Is there some acceleration for non-NN machine learning algos?
Random forests are the opposite: branching on conditional tests is very expensive. It's been several years since I last wrote raw CUDA and OpenCL, but if memory serves, back then the docs essentially said that every if...else amounted to running both branches on all the data and then deciding what to keep, effectively halving performance. So a decision tree just a few levels deep would slow you down by an order of magnitude.