This isn't inevitable. People do quit high-paying jobs to do something else. Early retirement happens.
For more info about this lifestyle, see this website: http://earlyretirementextreme.com
As long as you're done with your growth spurt, and have your personal style without need to look trendy every season, I see no need to try to buy the cheapest clothes. Affordable, sure, but without sacrificing quality.