On top of that, you have to remember that you can't have kids by yourself (at least not easily, if you're male): you need a willing partner. If you're working your ass off to save money for kids, that means you're either letting your relationship suffer because you're at work all the time, or you're missing out on opportunities to find a partner. And the older you get, the harder it is to find a good partner (really, you need to find her in college; if you haven't found her by age 25, you probably won't).
All in all, our society simply isn't set up to have an enjoyable middle-class lifestyle and have kids. You need to pick one or the other: either be poor and have kids and struggle financially, or enjoy a more financially comfortable lifestyle and forgo having kids. Also, having a happy relationship and having kids are at odds with each other, so again you have to pick one. Either have kids and look forward to divorce and child support or at least a miserable marriage where you dread coming home from work, or find a partner who doesn't want kids and enjoy spending time with her.
It's little wonder that the birth rates in developed nations have fallen so low.