But in order for it to be a useful degree you need to have math as supporting subject. This is the same as if you want to be molecular biologist you need to know a few semesters worth of chemistry otherwise you'd be given endless hard time, which you seem to understand.
I studied math at uni but because I haven't studied enough it always is so much harder to grok all important things in programming and CS. I would argue that it is a huge roadblock for anyone wanting to become any good at CS, but then again there's programming and there's 'programming'.