I've been "filling in the gaps" in math for almost a year now to learn machine learning stuff casually. I don't even need to use it, I am just obsessed with learning and I read about it for nearly an hour a day, and its still not enough.
Being self-taught at math introduces so many painful problems. If I were to do this seriously I would start ALLLL the way back at algrebra in 5th grade and work forwards ALLL the way up to linear algebra/calculus etc.
There's just too many tiny things and subtle details that are missed that I find. It makes any example require 10 times more brain power just to do simple things I don't remember, like the rules of factorization etc. So I'll go learn that thing which is simple, go back and 2 mins later I'm off trying to find some other simple thing. Mainly the idea of learning 5th grade math and such is so boring I just never have actually done it, so really instead of what I do, I would just learn the ENTIRE freaking thing.
Learning stuff like gradient descent and stuff is easy, but that's not even where all the hard stuff lies. I feel like trying to understand the math deeply behind those topics where youre not just glossing over explanation is where it gets difficult, and to do that you pretty much need the most solid math background without gaps.