I don't disagree that many people find personal satisfaction in their work, beyond seeing it just as a source of money. I do too.
Your point about time being zero-sum is a poignant one. Everyone wishes they had more hours in the day to spend doing things they like. But the question is about how we spend that marginal hour, and I think oftentimes it is better spent on family, friends or hobbies than our day jobs.