I'd feel much more comfortable with the idea that programming is math if my anecdotal experience of programs written by mathematicians didn't show that they have absolutely no clue how to write a good program - I'm talking about aspects of design like modularity, cohesion, separation of concerns, testability, sensible use of abstraction, etc. There's clearly something more to programming than math or there wouldn't be such a high correlation between good mathematicians being such garbage programmers. I suspect the whole debate is a sign that mathematicians simply dismiss these other aspects as unimportant and only count the mathematical parts and then come to the "surprise" conclusion that "wow, programming is all math!".