If you can find a perfect intersection, that's beautiful, but probably uncommon. Some of the most lucrative businesses are stuff like (anti)fraud/security/payments that is pretty darn drab.
One sure difference between these two things is that you absolutely know what you are interested in, but your opinions about what the market wants are going to be speculative until you have customers.
edit: i.e. judging what you think is beautiful vs. a Keynesian beauty contest.
As someone who has turned his interests into his business, it really can be tough. I have always loved infrastructure, and I have always loved maintaining certain applications and workload types.
On the one hand, you do start to understand after awhile what the “MVP” looks like. On the other hand you can eventually fairly well predict what your top feature requests will be right out of the gate.
Integrating the two effectively while actually getting something out the door can be a real challenge. I have criticized many firms and offerings for not having “basic feature X,” but as someone who is building a product that’s effectively never complete, I’m sure someone could say that to me too, about at least a few things. Some come immediately to mind.
And I’m still sort of surprised at times by what my own customers do and do not want or ask for, or even care about. When my interests are aligned with theirs, the product/service can move forward really quickly. When they’re not, it can be a struggle to iterate effectively.
For example I found having to seem like a good employee to my boss by looking busy and not leaving before at least 6pm a sort of stress I don't have now that I don't have a boss - I don't have to pretend I'm working at least 40h/w if I manage to do what I need to in less and go pick up my kids from school in the early afternoon some days.
At the end of the day although I'm more stressed in some ways (and less in others), I've never been as happy as an employee as I have been running my own company. You can even learn to enjoy the new aspects of your work that have to do with running a business and adapting to the market.
Like you I also learned a lot about aspects that I never understood as an employee and I'm now a lot more careful before judging other people's actions without having all the context (sometimes it makes sense from the inside)!
That feels like a reasonable approach to me, though -- as with anything -- you need to know to pull the plug as soon as you have some confidence that it's not going to work out, which might be harder to do for a hobby, since you presumably have some emotional attachment to it.
> Some of the most lucrative businesses are...
You don't really need to aim for "most lucrative". If you're trying to promote "hobby" to "business", I think you should just aim for "good enough to pay the bills and give me a comfortable lifestyle without working myself to the bone". Starting out with a goal like "I'm going to turn my hobby into a blockbuster business that gives me FU money in 10 years" is probably unrealistic, but that's ok. More modest goals are just fine.
Nobody wanted an open-source Unix-like OS, a keyboardless touch screen with third-party apps, a mini hard drive with a display and headphone jack tacked on, maps that could be infinitely panned instead of clicking arrows, etc. But they ended up being some of the defining products of our time.
The worst case of working on your own thing is that you make something nobody else wants. The worst case of working on something you think other people want is that nobody wants it, and you get burned out.
IMO, focus on the problems - those often exist before the solutions have manifested.
This is it.
This is how I approach in writing software.
99.9% of the stuff I write is going to have one user. But it's one more than zero, and in the 0.1% of the cases where it does have >1 users, well, I call that a success.
Do what the Market wants means making a tradeoff in favor of earning more (and becoming rich(er)).
Doing what you want to do means making a tradeoff in favor of your happiness. You will perhaps earn less, but what you do is more in tune with your values.
A tradeoff is not always required and both of these motivations may become synergies, but in the authors case, I think this is what happened.
I do agree 'what you are interested in' is often wrong, that's a good starting point for ideas but after that you need to see if there is an actual market there.