At some point, I narrowed a list of 40+ idea down to 4, and then I picked one. I used a somewhat formal process to help me out, and that was to evaluate each idea based on a checklist of 7 requirements:
1. Do people need this? (i.e. have I identified a niche that I'm very confident will love what I build)
2. Can I personally do a good job marketing and distributing it? (i.e. do I know where the target customers hang out online and am I sure I can get the word out about my product there)
3. How hard will it be to avoid churn and keep retention high? (i.e. is the product set-it-and-forget-it, or at least somewhat naturally re-engaging? or will it require lots of effort from users' to keep using)
4. Is it meaningful and beneficial to the world? (i.e. will I enjoy working on it for years to come)
5. How long will it take me to build an MVP? (i.e. do I have time for this now)
6. What kind of effort will it take to keep it running long-term? (i.e. will I have time for this later)
7. Can this idea actually generate revenue? (i.e. do I have a business model in mind that has been proven to work for this kind of idea in the past)
My advice to you is to come up with a similar checklist, run through your ideas, give each a score, and go with the highest one!
It's generally easier to optimize this path, then to "pick the right idea." Because it's basically impossible to know ahead of time. And, besides, you'll learn a lot through each iteration. Good luck!
Recently I gave up on a AVR emulator I was writing (in C, to learn C at the same time). The core controller structure and instructions were fun, but when it came time to implement the IO, I realized I just wasn't that interested. Plus I have older kids now who realize when I'm neglecting them, a bigger house to maintain and a more demanding job.
Now I just do a few algorithm exercises in Juypter or crypto challenges in Go each week. I'm still learning and I get a sense of accomplishment without the nagging unfinished project aftertaste.
1. Pick something you can do in a relatively short time, say less than one year. That way you do not run out of steam working on it.
2. Pick something that you will learn a new skill or skills working on. Regardless of if what you choose succeeds or fails, you still come out ahead with a new skill.
The upside is that you can find a probably that people are provably interested in paying money to solve. The downside is that most ideas you'll find are going to be a real grind to solve and are unlikely to be passion projects which is often helpful for success.
I could hardly spot a good "pattern" in the jobs posted.
Most of them seemed ad-hoc and custom enough that after looking at a 100 you will still struggle to find a common denominator.
I believe this post was the genesis of the odesk idea.
SaaS tends to be inexpensive compared to paying a human to do something, even if that human is underpaid, since costs are amortized across many customers. You're paying for something more similar to (CPU time+profit margin) instead of human time. I'd also hasten to add that there are two sided marketplaces where average costs are higher.
Try to reach milestones on any project. Personally, in this stage of my software engineering I almost always intend to ship projects with "production" quality to "production".