Because we're social animals that have evolved to weight appearance very highly in decision we make. I'm not trying to justify it, but it's a fact. We're not vulcans, and even if we we're we'd recognize that what we find attractive has evolutionary significance and weigh that signal accordingly.