IMHO many people struggle with maths because they're not used to the fact that you have to read maths differently. You can't just digest it quickly like many other subjects. You have to ponder over definitions for a while and play with them and try to come up with examples and counterexamples.
That, and I think like reading, it takes a lot of practice, and that practice is really hard before you are good enough at reading to see the story. I think I'm at the point in math where I can't read it well enough for it to tell me a story, so practice is a real slog, just like it was for reading at that point.
If you're having trouble with specific material (e.g. calculus), sometimes the reason might be that there's something else that you don't quite understand well enough yet that is a prerequisite. In such a case, the first step would be to identify what that is (it could be unfamiliarity with trigonometry or a lack of skills in algebraic manipulations) and then go back and refresh that.