We don't know what the default state of human attraction is because we don't have access to a control population of human beings without the same shared societal history of the rest of us.

Which is to say, it's incorrect to broaden your personal attractions to all men, and especially incorrect to try and couch it in some argument of biological naturalism.