It's not that I can't understand it. I don't have that urge personally but I can see where it comes from and if that makes you happy go for it! The reason I personally get so defensive and anti side project is because as someone who's very defensive of their non-work time (due to spending it on other productive hobbies that bring more joy into my life than anything else) I've experienced a lot of colleagues/managers who have tried pressure or belittle me into doing side projects. It's been implied that I'm not a 'real' engineer or that I can't be successful in my career if I don't. And frankly I resent that attitude because it's bullshit and it tries to take away time from the things that make my life truly joyful.
As you say do what makes you happy, side projects or not. Unfortunately a lot of folk on the side projects side don't/won't accept them as optional if you want to be seen as a competent engineer.