"Rather, prejudiced people take one look at you, see your gray hair, and make assumptions about you""

Often, I suspect, it isn't even about making conscious assumptions. People just have positive and negative subconscious reactions to certain physical traits.

It's striking that you virtually never hear any discussion of "height discrimination", which I put in scare quotes because it barely seems to a recognized concept in the world of work (although it is somewhat recognized in the dating world). Yet there's lots of evidence that tall people have systematic advantages in life. They're seen as leaders, more likely to become CEOs, more like to be elected to political office, etc.