I work at an A-round startup. Tons of people in FP&A, sales, accounting, and HR get promoted every six months. Engineers rarely get promoted. No refreshers. No raises.
Guess what? Engineers are now clocking out at 4pm, they are working on side projects, and it is the most rational thing to do.
On the flip side, I've worked at places where engineers are rewarded regularly for outstanding work. It makes sense to focus on your job.
On size advice does not fit all.
That one is actually pretty simple: pick the one that you have the most to learn from, because that's the whole point of doing a side project if you are trying to leverage it as something that would help your knowledge/career development (as opposed to doing a side project for some supplementary income, for example).
And yes, I am aware that there are many unknowns, and when you are trying to pick a project with the goal to learn the most, sometimes the projects where you think you already know 90% of the solution end up being the ones where you know the least and have more for you to learn than projects that had initially more unknowns. But without the ability to know the future, picking the one that has the most "stuff to learn" (according to your first assessment) at the initial discovery phase is imo the most optimal strategy. It is fine if you discover later that the other project had a bit more to learn than the one you picked, because the goal is to learn a lot averaged down over a long period of time.
Usually there's one project that you just can't get out of your head. If you don't know which it is, try to take a break from all of them, and you'll quickly learn which one maters.
100 side projects is the chance at one idea worthy of implementation. If you have even one good idea, all of your bad ideas weren't only worth it, but amount to how you got there.
One way is to have hobbies, side hustles, volunteer work, whatever. As long as you're doing something that requires solving problems that are different from your day to day programming job, you will find a constant stream of problems that maybe you can improve with a bit of code.
Some of my side projects are directly related to my work, but most of them aren't. I'm working on something right now because someone in my life needed advice on digital security, and it turned out to be not an easy problem to address.
Somewhat relevant, PG has a post on startup ideas http://www.paulgraham.com/startupideas.html
The hard part for me is to find the time to implement an idea. An idea for me just naturally flows from a problem.
Whenever I get an idea, I quickly note the main pointes in a note-taking app and usually never return to it, as the secret of creating actual useful products is to spend the time to create them well. Ideas are worthless if they are not well implemented.
Friday: I can build a nice and simple API for OpenWRT that is API focused and provides easier access to the common functions that people want from OpenWRT, this would help mobile app developers create apps that talk to OpenWRT in a simple way. Let me just write that down in my notebook.
Saturday: Let me just sketch out this new design for a workshop cabinet that will let me combine my tablesaw, bandsaw, chop saw and tool storage in a single location.
Sunday: I want my security camera to combine all the detected motion in to a single multilayer video that lets me scrub through events like layers playing simultaneously rather than scrub through time. Let me just write that down.
Monday: OpenWRT should have a "state" that is queryable via a curl request that determines if OpenWRT still needs to be configured. I can build that as a simple API and package it. Let me just write that down in my notebook.
The ideas aren't the problem.
Started a new side-project on the weekend (after a long long hiatus of just focusing on day job). Phase 1 was simply getting the stack up and running (PostgreSQL-backed, Python gRPC service taking gRPC-web through Envoy to a React front-end). Took some fighting through the setup to get a full-stack existence proof, ideas that arose during the working session:
side-project TODO #1 - Write a blog post about getting this all setup using current versions and push a public github repo with the setup to help other out.
side-project idea #2 - Build a micro-PaaS where a customer can define a gRPC interface, Python module implementing that, and with a single command, push this to a live serving endpoint.
side-project idea #3 - I need a state machine as part of my application logic, TODO: write a Python non-ephemeral state machine library that, with a simple Python internal DSL, backs state machine persistence onto a PostgreSQL table, handle automatic schema updates as the state machine is changed / versioned.
Literally not enough time to do any of these things though :)
re: gRPC, largely pragmatic because of familiarity with protos and RPC interfaces defined in terms of protos. I wanted a typed API definition that generates first-class implementations in Typescript and Python.
Haven't had much success in the past re: using other formal mechanisms of defining API interfaces, and I'm no longer up-to-date with what's recommended in this space (any suggestions).
Alternative was GraphQL (since using React) but that's a learning curve and dealing with protos in Typescript and Python is a known-known for me.
I read that somewhere, possibly here. Maybe someone can link the source, if they have it.
I often think I should forget about this project altogether and focus on a viable one.
The alternative is to wind up as Burdian's Ass, who is equally hungry and thirsty, can't decide whether to eat hay or drink water, and so dies of hunger and thirst.
And if you choose wrong, you'll find out soon enough. Personally, I had a lot of projects I couldn't stop thinking about, but when I finally started working on them, I quickly realized it's a bad idea. The usual reasons were: a) I already found it done by someone else in a form that satisfies my needs, b) it became obvious that either the process or the outcome won't be as satisfying as I thought it would be, or c) after scoping it out, I realized I'm not in a position to invest the required amount of time and effort.
Abandoning a side project early is not a bad deal. Your research and thoughts put into it stay with you forever, and it often happens that you'll get secondary value from them on some other endeavor.