We have an old saying "te rafte pika e diellit", may you get a spot, sun spot, aka cancer from the Sun.
People knew excessive sun is unhealthy.... if you could you avoid it during the summer. It was not necessary to make you look 'rich'. Ability to avoid it, meant you were rich. And people wanted to be pale, because you look rich, that is a second order effect, that came because of the original reason (aka, too much sun is unhealthy).
I think you are retro analyzing and presuming individual intelligence because this collective result looks right to you. Really, some people understand some risks and many people who can follow taboos they can afford are not exactly sure what they would learn first hand and what part is superstition.