I'm in a similar situation, although with only 1 kid yet. I think you're spot on that building a working prototype in your free time will be either slow or exhausting.

But most of this start-up advice seems to be geared at single college students, and they'll have enough free time to pull it off.