What I saw was some very intelligent people whose hard work, creativity, and genius I admired making very little money, scavenging for grants like jackals, their spouses working second jobs, their clothing a little worn, their cars breaking down even as they labored over Big Issues. That is the payoff?
I did not pursue it any further. Yes, the degree itself impresses people (sometimes more than really necessary, is my feeling), and it certainly primes you to be able to switch gears for entirely different frameworks, but it has opened doors.
Meanwhile, I occasionally look at the odd paper, sniff around. For strings (oh, not strings, superstrings) (oh, not superstrings, M-theory) it's all out of my league, but not so much so I cannot feel the shape of things and just how far they are from being more than just very abstract mathematical constructs. "Fiber bundles," exactly how many strings and shiny hag-stones are we out on that particular Glass Bead Game?
I accept the possibility that the universe may simply be too difficult, on a mathematical level, for mere humans to understand. It could happen. On the other hand, I suspect that the development of mathematical models of physical phenomena without frequent touching of the "real world," fumbling for guideposts, why, you could very well wander off into a paper map of your own creation, unrelated to the world you wanted to explain.