If monogamy is valuable, and I think it is, then most men should be parents as well. I can’t imagine a world with a few guys fathering most of the kids, that seems like it would cause lots of problems.
The whole idea that child bearing should be perfectly and equally distributed across the whole population is a construct of the social welfare state, enabled by subsidies, and is just an artificial form of equality like any other. As such I think it should be examined and debated like everything else, and not taken for granted as self-evident or "natural".
The median is close to the average in kids per women in the developed world. Having four kids is harsh on many women’s bodies; eg my grandma had 8 kids, but she died in her late 50s. While no doubt some couples will want to go for four kids or more, I don’t think it’s ethical to encourage it over a more standard 2 something for most couples.