I like the way you think, but it's not two things you can compare. For once, living in decent conditions (3 meals a day, a roof on top of your head, ...) ranks much higher than dating and reproducing on Maslow's hierarchy of needs.
Also, the gene pool is quite diversified and most people can find a match during their lifetime. It's not common for attractive people to have multiples spouses.