I see these sorts of contracts at a decent number of jobs, but I wouldn't say the majority. Every time I've been presented with one, they've been fine adding an exemption for my hobby work - as long as it's distinct from my professional work. I believe California requires such an exemption - I've gotten a fair number of contracts where it was already specified that they only owned the right to code that was related to my professional duties.
In your case you're talking about a specific technical hobby which is also cool. Just code all of it on your personal comp not during work hours. Once it get big hire a lawyer and get a llc. Don't limit yourself, identify problems and find solutions. Be an engineer. You got this.