Work is not misery as long as you believe in, and are constantly engaged with, your work. It becomes misery when it doesn't align with your needs and expectations.
The biggest question is, what would you rather do? What would you do with your free time if money wasn't an issue?
If you are happy with your product, then what is it about the work environment? Does simply managing people stress you out? Or, is it the expectation that "I should" enjoy what I do because it is successful?