What if you have young kids? It might be worth the suffering for their benefit.
Our society says parents can make whatever choices they like, but having children is one of those places when we should be thinking of others (the kids) not ourselves.
I decided against having kids when I reached 40ish for the above reasons (along with some other reasons).