I couldn't disagree more. I started two side projects while working a full time job. I also have a family and coach my kids sports team. I enjoyed working on them in the spare time I could find, enjoyed seeing the progress I made (no matter how slow). Because I couldn't dedicate a ton of time working on them, I had plenty of time to -think- about it. Driving, grocery shopping, in the shower... when I did sit down I knew exactly what I needed to do and was very productive.
After a couple of years they both had grown to a point where I was able to comfortably quit my job to pursue them.