Look into sparsification of ML models: https://neuralmagic.com/ If your cpu has avx512 it can make it even faster.