See Bateman's principle [1]. In most species, variability in reproductive success is greater in males than in females. This is also true for humans (even today) [2], modern human may have twice as many female ancestors as male ancestors.

This is fairly intuitive result when you consider gestation time - imagine a scenario where there are more male than female in a given moment for a given species. At most each female got fertilized once, so the number of distinct male parents is less than or equal to the number of females. If there are more females than males, the you get the same situation (less distinct male parents). This result is simply because a male parent can fertilize multiple females quickly, while gestation period limit the females.

This dynamic has led to sexual selection mostly affecting the males of the species, which means that this is where the competition happens. The move from polygamy to monogamy in human culture has allowed more males to reproduce and reduced this effect (although it still exists, and is significant).

1. https://en.wikipedia.org/wiki/Bateman%27s_principle

2. https://en.wikipedia.org/wiki/Reproductive_success#Humans