Nah. It's not hard. Just don't buy stuff you don't need. Good ad blocking helps, too.
I know a couple who were way too much into bling. They blew through $15 million and ended up in crap jobs. The wife once told me I should get a Rolex. I said "Why?".
There are many people who have money but don't show it. Read "The Millionaire Next Door".
You could attribute that to humility or good manners, but you could also attribute it to the fact that their persona is good enough of a signal that they don't need to add another signal.
This is especially true of traditional nobility and the super-rich. It doesn't matter whether Bill Gates wears a $40 watch or a $40K watch because everyone already knows that he's one of the richest guys in the world. Marie Antoinette wore diamonds not because she felt a need to demonstrate her status and power, but because she liked diamonds and happened to have the status and power to get them.
https://www.washingtontimes.com/news/2014/sep/30/bill-gates-...
However if you appreciate a fine watch that does more than tell the time (and who's to dissect the whys and wherefores?) buy it and support the families of the workers who are part of the company that produce it. Someone sweeps the floor in their workshops and needs the job. Ditto all the other luxury goods.
Buying a $20 watch will support some Chinese workers. Buying a $2K watch will support Japanese or Swiss workers. But I don't think buying a $20K watch will support workers any more than a $2K watch does. It probably depends on the company and the watch, but I think it's a safe bet that most of the markup at the extreme high end of the spectrum goes to a small number of managers and dealers.
Designing and making a mechanical watch with many complications takes a lot of time, so it's entirely possible to buy even a 200k watch where you are mostly paying for the actual labor (of a high-skilled professional/artist).
If there were no idle rich to make expensive products for, (say they were all beamed off the planet to work in an alien slave mining facility) the end product that was produced from any job would defacto have to help those in the same or lower class of the worker, benefiting everyone.
In other words, what's better for the world, yet another rich man's yacht, or cheaper, better tasting loaf of bread that a poor person can buy and eat?
It's simply a matter of people wanting to live (or imply that they live) a lifestyle that is beyond their means, because living a lavish lifestyle is a indicator of success in life, thanks to our consumerism-based society.