You speak as if being a grunt doing webdev isn't a good gig. I have a degree in chemical engineering and do webdev stuff, but I know I'm not doing computer science nor do I care to, I'm content making good money doing easy work.
Even if you are, the industry advances, and that kind of grunt work has a history of being automated away. Web masters probably felt the same way.
You don’t need to get another degree, but since you have the math background, it wouldn’t be that hard to spend some time on CS fundamentals.