This assumes all men would be equally likely to get to have sex with the women. Otherwise, if the whole society has to take care of the children, but only a few men get to procreate, it just means many men working for free (like drones) to support other men's children.
Of course that would be a good scenario for some members of society - the women who can choose to mate only with the most attractive men, and the few lucky men. I doubt it would automatically create a happier society, though.