Let me give you an example of how referring to you in an insulting way can change your attitudes and discourage conversation.
1. Words have a lesser impact on you because you've [a lucky person] who doesn't live a life were they have been used to deny you opportunity and basic humanity constantly.
2. Words have a lesser impact on you because you're [a white male] who doesn't live a life where they have been used to deny you opportunity and basic humanity constantly.
Looking around the internet, #2 certainly seems more contentious. But let's be honest here, the overlap of case 1 and case 2 is much more likely in most of the western world than other ethnicity and gender and sexual orientation we could put in there.