This whole argument reminds me a bit of the book, Godel-Escher-Back, which is (partially) about the interplay of formal and informal systems. Both are necessary, and one without the other tend to be useless or sterile.

Sometimes, when I've asked for an reason/explanation about some physics problem, I get a response like: "because of this equation". And in my brain there is a mismatch, were it doesn't feel at all like an actual explanation, because abstract symbol manipulation feels like, well, symbol manipulation.

But I think that's the wrong interpretation, because someone who has been really deep in mathematics for a long time feels the math at a deeper level than I do. For example, it's like explaining something that's true because F=MA. Before I had any physics knowledge someone answering a question with "It's because F=MA" would have felt off, in the same way. After doing a bunch of problems then F=MA is a perfectly fine answer. They are just a lot better at getting meaning from the symbols and following the manipulations.