I think MySQL was not the correct tool after all. Click house is a lot more optimized