This is extremely well articulated, and is exactly true in my experience.

People take gossip as "gossip"- an unrealiable approximation and gross-oversimplification of an individual. Thus give people with poor reputations the benefit of the doubt, believing themselves to be open minded. (which is true).

When they are shown just how 1-dimensional some people can actually be, and that the gossip depicts people precisely, it can be jarring. Especially as, like you mention, many people are very personable, charismatic, charming etc;