If you're by some semantic classified as unattractive, you will automatically feel segregated and marginalized. This is not like sexual orientation where non-heterosexual people actually have a desire to express their sexuality but this desire is repressed by a regressive, judgemental society. This is exactly why "Gay Pride" is a thing.
Compare that to being unattractive. Unattractive people, whatever that means, don't want to be part of that group. That's a pure factual reality of the human instinct. People want to feel attractive because they have a natural instinct to intermingle and reproduce.
But even if we opt to oversee this particular deficiency when attempting to justify an "unattractive" protected class, the problem is that "attractive" is extremely subjective and usually dictated by societal norms.
What's attractive today wasn't attractive 100 years ago. What's attractive in Africa is not what's attractive in North America. What's attractive for me (even in this dictated herd mentality) is not necessarily attractive to you. The only common factor that all these perspectives share is that nobody wants to be classified as unattractive.