At one time I was in a physics program at university until I discovered that I couldn't understand physics. I changed to computer science ;-). My experience, though, was that most people I talked to had only a cursory understanding of actual physics. What they really understood was the math.

Like I said, I sucked pretty badly at physics but my impression is that people tend to jump back and forth between imagining the physics, modelling it in math, making predictions based on the math, imagining again what the physics would look like, ad infinitum. So for complex problems, I don't think anyone has a whole physics problem in their head at once and are reasoning about it at that level. They just have pieces and use the math as a way of swapping parts of the problem in and out of their head (if that makes any sense).

I think that's one of the biggest problems with popular descriptions of physics. It doesn't make any sense without the math, but average people don't have the math background. So the descriptions make absolutely no sense. Often you get people who are interested in the physics (but without training) imagining all sorts of crazy things because they have built their understanding on these kinds of nonsense descriptions.

It's been this way for quite a long time. Newton had use new mathematical techniques (calculus) to understand things like the force of gravity. Even if you go back to Archimedes, I wonder how much of his reasoning about density was based on modelling it with math and how much was based on intuition about the physical universe.