It's never felt like math to me. If it did, I probably wouldn't be able to do it. I'm not especially good at math and feel damn near dyslexic when I try to read math-notation-heavy writing. Programming, meanwhile has always come easily and naturally.
... except the languages that try to look like math. Looking at you, Haskell. I get the concepts just fine, but I can't stand the style.