You started by making the point that he might have more money, but you have more time, which you conclude is ultimately more valuable.
As many have pointed out, he can sell his business and retire, which you don’t have the option to do. Your rejoinder that he can’t do that because then he’d have too much time makes little sense, both because you’ve already setup the axiom that “more time = more freedom”, but also because he could at that point decide to do all kinds of things: start another business, buy one, get a job, work for charity, whatever. He’s quite obviously more free than you, although perhaps not psychologically, who can say.
You don’t need to defend your life choices to anyone, but from the outside it looks like you’re making an illogical case because you can’t accept that he actually does have more freedom than you. Which is OK! There’s always someone out there doing better than us.