One term encourages the idea that they're people first-and-foremost, with the adjective used to describe a particular subset of people.

The other removes the emphasis on them being people and is pretty depersonalizing.

As you can imagine, it's much worse when it's a historically marginalized group--as an example, fewer people will care if you say "blondes" vs. "blonde-haired people".