There are people whose job it is to make pretty math, they are called "mathematicians" ;-) So much of early physics, be it thermodynamics, electromagnetics, classical mechanics, is genuinely beautiful and elegant. I can see why quantum physicists would strive for that ideal as well. But at the end of the day I agree that physics should favor creating testable predictive models over developing untestable, elegant ones.