As a computer scientist, software absolutely is math. Software is not bound to CPUs/etc.; you can evaluate algorithms by hand with a pen and paper if you want to, and you will get the same answers (maybe more slowly, but likewise if you use a slower CPU or a non-optimizing compiler). Lambda calculus is without any doubt a field of math; any code can be represented by equivalent lambda expressions and you can convert to lambda expressions automatically (a common compiler strategy for functional languages).
Yes, C and Python do not look like the expressions in a typical algebra textbook -- but algebra does not look like calculus or geometry either. Representation of your algorithm is irrelevant; as you say, it is the wrong level of abstraction.
Oh, also, in case you were thinking that there is some important difference between software and math, here is something to think about:
https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
(Executive summary: all (terminating) algorithms are proofs of mathematical statements.)