The idea is the consequence of noticing that there is a problem; a gap between a current state and a desired state, and thinking about closing that gap. In other words, starting with a problem is more conductive to finding ideas than focusing on finding ideas.
Finding problems requires that they be considered as such, that there be a desired state that is different from the current state. That there be a gap (error). At some point, a hunter-gathered said "You know what, I'm kind of tired of hunting game. I'd rather chill and have these animals and plants right here". His current state differed from his desired state, and there was a problem that was born. That was a different problem than the state of being hungry (Current state: hungry. Desired state: not hungry) which lead him to hunt and gather in the first place.
Finding problems has a lot to do with exposition. The more situations, the higher the chance a problem hits you. Problems may not be yours but other people's, and they may not be aware they are problems (current state ≈ desired state. Close enough that they're not picking up the error, or that they perceive they can't do anything about it, often due to asymmetry of information).
That is to say, focusing on problems is more conductive to ideas.
Then who does the problem belong to? Your problem or other people's. Sometimes when it's other people's problem and you like these people, the problem becomes yours. "Fixing someone's computer", for example. Sometimes you go to use their computer and it's horribly slow even if the specs are good, and they might even have not noticed it or they got used to it or it's not that big of a problem until you fix it and they notice just how fast it's become (back to when they purchased it, but they forgot how it felt - perceptual contrast).
That is to say: what's your target audience. What's the market? The segment. Do you like that crowd? For some people, it's not important, but it is for me. There are categories of people I could talk with and topics I could talk about for hours and days and not feel like it's work (previously MLOps platform), and there are others I wouldn't really hang out with by choice or default (helped a friend making a product in the dental space).
Then there's qualifying the problems: Is it a real problem? For whom is it a problem (buyer, user, etc)? Do they know it is a problem or will I have to make them see it (raising awareness and "educating" is expensive)? Has it always been a problem or just now? Why now and what changed? When was the last time they had that problem and what happened and how frequently they're having it (frequency, impact)? What have they done to solve it (time/money/people/etc)? What were the outcomes (are they still dissatisfied)? Is it urgent/mandatory? Is it expensive?
How can I search for and reach the people who have that problem in order to learn?
>I'm desperately trying to find an idea I can work on, taking advantage of the current ease of developing MVPs.
Either hang out with people and listen. There are a lot of times where people describe something frustrating for them that is obvious for you. Now, whether the solution already exists and it's simply a lack of awareness, or they're aware of solutions and none did it for them, or that there are organizational hurdles, it's up to you to figure that out, but you're often in a position you can see it.
What's great is that you got the technical part down; one fewer layer of risk is good; but in my opinion, most of the time it's not that that kills an endeavor. The code cemetery is full of MVPs and full-blown products that found too few or no buyers to make viable at the time and place.
>but everything I create or try to do seems like there are already dozens of other solutions doing the same thing (specially in the fields of wrapping AI).
Kind of like beverages, snacks, clothes, cars, phones, laptops, languages, frameworks, etc? It tells you the importance of market, distribution, and a bunch of things other than code.