There are certain types of work that I don't have the mental energy to do when I get home from a full workday, or at least not to do well. And, honestly, if I try too hard, I risk lowering the quality of my work the next day and burning out completely at some point. And burnout is a fairly large reason that startups fail: "But I think the underlying cause is usually that they've become demoralized" (http://www.paulgraham.com/die.html).
There are other types of work (e.g., practicing piano or voice, two pursuits where the amount of time spent is pretty steadily and monotonically correlated with success) that I can do just fine. It seems more than likely that, if we get it to the point where you can do a successful startup on 5 or 6 hours a day, spending an additional 5 or 6 hours per day on the startup is unlikely to make you more successful, but those same 5 or 6 hours can be productively directed towards music practice, raising children, or whatever else you want to do.
(There's also a line of argument about parenthood being of intrinsic importance, above success at startups, success at music, etc., but I'll let someone else argue that point.)