Not really. Mostly just algorithms and procedures of some math concepts(Calculus and Linear Algebra) which roughly corresponds to the first year of North American math major.
Besides, I never heard of math majors taking cryptography that early. Usually, Intro to Real Analysis, Abstract Algebra and Abstract Linear Algebra come first.
In contrast, I know engineers who live and breathe PDEs and tweak compilers to solve them faster.
I doubt my CS professors would be able to solve a PDE..
Beyond the basics, not even many math professors can do that. Math is too vast and people specialize. Strong algebraic geometers are not necessarily strong analysts or algebraists or logicians.
1) No such thing as universal mathematician in this day and age.
2) Engineer's PDEs(algorithms) are not the same as mathematician's PDEs(theory). Same as comparing a student in China who learned English to communicate with English speakers to English majors from English speaking countries.
So yes, if you take a Computer Science major with a focus on Software Engineering, you will not learn enough math to minor in math "for free". If you pick a more mathematical subfield to focus on, you should probably declare a math minor for the one or two additional courses it will take you.
I made a truly stupid choice: I graduated in 7 semesters with a Comp Sci degree concentrated on PL theory without picking up the additional courses for a math minor. As a result, I'm "condemned" to learn that material independently later on. "Luckily", the Technion required me to do extra coursework for my MSc, so I've had to buck up and learn more theory.
Now get off my lawn until I'm done with my highly theoretical machine learning exam ;-)!
If you want to do good at machine learning...electrical engineering is probably a better choice; the maths learned in EE overlap fairly well with what is needed to do ML. PL theory is quite niche, even for PL researchers.