I hear about this argument and I can't help myself wondering:

How many short men are getting excluded from gene pool as a result of this?

My take: almost none. They might get "less desirable" women, but otherwise all men and all women are going to pair up.