I am also on the same boat. Based on my learnings and experience from running failed startups, here's a system I am experimenting.
Step 1: Observation
There are oppurtunities around you; some people are good at finding them and exploiting it to build profitable ventures.
The intended outcome is to find a problem people are struggling with and help them solve it.
Here's how I find problems.
1. Most Voted "Ask HN" Queries: I have already noted your question as a story idea for my medium blog. Since this is something, people need help in figuring out.
2. Comments Most Voted HN Posts : Not all posts have comments where people talk about their problem, but some post has insightful data.
3. Comments on Top-Listed Products on Product Hunt : Find out the top listed products on product hunt. Go to the comment section and check out for the 'cons'.
4. Conversations : Go out and talk to people who have a different perception of the world. Try putting yourself into their shoes. Then try to if you can bring a solution to their problems with the knowledge you have.
5. App Store Reviews / Amazon Reviews: These reviews contain problems with current product users have.
Step 2. Study
Now that you have got the puzzle, you need to find a cheap and efficient way to solve it.
I mostly try to see how people in a different field solve this problem. Later, try to see if their solution can fit into the issue I selected.
I also find solutions in books, research papers, hackathon projects, etc.
For now, you only need to find a theoretical way to solve the problem.
Step 3: Shadow Testing & Writing
Make a landing page about your ideal solution. Make sure you include good graphics such that your product looks ready.
The landing page also won't be enough to attract people. Write about your learnings on Medium or your blog. At the end of your blog add a signup link to your landing page.
If you are new to copywriting, please read the book "How to Write A Good Advertisement" by Victor O. Schwab.
If people can relate to the problem, you would be receiving signups. Ask them to join a slack or a telegram or a WhatsApp group.
Step 3. Validating with the Early Adopters
These people are a boon. They have extended me so much help in my earlier startup. They do this because they would like to see their problem solved.
Start having a conversation with them, build prototypes. Incorporate their feedbacks. Make sure what they are willing to pay exceeds the production cost so that you can make profits.
I hope this helps.