If better tasting drinks really did make you happy--something I don't believe--the rational thing would to be to brew your own coffee or tea. This has the added bonus of being a learning experience. The $5 latte drinker, at most income levels, is someone who hasn't evaluated his incentives rationally.
Thinking rationally about money is a muscle. Negotiating a $5000 salary increase then buying $5 lattes is the equivalent of going to the gym, then driving a ten minute walk home.
I could understand someone that tells me his 5$ coffee tastes "better". But if you drink your 5$ coffee just because you are feeling stylish ...
If something makes a person happier, it makes them happier.
Or I could spend $1 or less on my morning coffe that doesn't taste as good, and not wake up in the same, refreshing way, and be overall less productive.
Oh, and I still manage to save 20-30% of my overall income (depending on the month). Lattes don't matter. Overall spending habits and and fiscal approach do.
For instance, saving 50 euro on airline travel ... not worth it if you can afford the extra 50 euro.
But you have to be perceptive enough to see you are "buying" a retirement, not "denying" a Latte.