You can't possibly know whether it has impacted you as a developer, because you don't know what "you with more math" would be like as a developer. You can say that you're doing fine in spite of no math background, but I don't think you can reasonably claim anything about where you would be if things were otherwise.
For myself, I have experience programming with no math at all, with high school math, and with self-taught axiomatic math (I've been programming since I was quite young). I can say with certainty that every time I've learned more math, I've been able to improve as a programmer as a direct result. This even applies to relatively esoteric parts of mathematics, like category theory.
I don't think anyone's claiming that you can't work as a software developer without a heavy math background. We're just saying that you can go a hell of a lot further if you take the time to learn the math.