> When I'm old I'd rather say I'm broke because I lived my life to the fullest
That's probably where people differ - how they understand what "living life to the fullest" means.
To the OP, it means spending less on what he does not need and working on the things he likes. To others, it's travel, meeting other people.
There's no shame spending your money the way you want. Just don't live beyond your means. Know your (financial) limits.