Well there goes the correlation/causation angle. I'm sure everyone has experience with seeing someone for the first time and thinking that they're incredible attractive until they open their mouth. The mind builds perception out of many signals and ocular vision is just one kind. Typically 'ugliness' is the presence of unwanted phenotypes, and in the absence of that 'ugliness' it's very easy to impute beauty.
I also think the idea that beautiful people become more likely to believe in a 'just world' is a strange way to think of it. If we're assuming a baseline level of selfishness, evolutionarily it would make much more sense for ugly people to try to shift consensus away from a 'just world' as doing so is likely to increase the amount of resources they receive. That is, instead of beautiful people recognizing their own privilege and thus lobbying for more, to me it makes more sense that people who don't see themselves as beautiful will tend to lobby for more beauty-blind practices. No one complains the deck is stacked when they're winning.