So you do what I did - marry a woman with a good career and become a stay at home dad. Even better if you have a skill set that allows you to do some work from home, around nap times to allow you to keep your eye in.
In your case it is because work gives you happiness and totally nothing to do with material things.
Majority of women live without expensive holidays and expensive cars. Majority of men live without expensive car and men are more car-focused and tend to be the ones who control the car when car is in family. Most people, neither males nor females go to expensive restaurants.
You know what else second income does? No one can tell you that he earned money and you are wasting them. It means that you can leave when relationship goes bad. It means you can handle when husband goes sick or dies. It means that you are less vulnerable and safer.