I think you're right, society is not homogeneous and language does evolve differently in different niches. Usually this can be observed in smaller groups, which have a strong pressure to "code switch" the dominant language of the society where they live. Minority group members do this all the time, all over the world; often there is a continuum of language/dialect/register that speakers navigate throughout their daily lives.
What's peculiar here is that american society has become polarized, on a rough 50/50% split, along certain cultural norms that affect a small part of language.
Members of group A will not easily concede "code switching" to the vocabulary that is ok for group B because group be is not clearly the majority dominant group.
In this case it's easy, since the word has alternatives, so a member of group A doesn't have to code switch to group-B-speak and say "retard" instead of "idiot".
Furthermore some vocal members of group A will demand that members of group B refrain from using that word on the same ground (there is an easy alternative).
You may be rightfully annoyed that group A is dictating something to you, dear group B member. A meager consolation is that there many minorities in the world that know how you feel (although some may not identify you as a minority since you're not; but that's beside the point, that's how it feels to be one).
Culture is complicated.
Where I live there is a big chunk of local culture that uses blasphemy as casual filler words. Other areas of the country, and a sizeable chunk of the same region, find that utterly repugnant and it cannot be used in polite speech (e.g. people are fired for saying "porco Dio" on television). Locals here just know it when it's ok to use it and when it's not a good idea. Some insist they should be free to stay whatever they want, and "porco Dio" they may well be right! A famous local nobel prize winner (Margherita Hack), when asked "do you use blasphemy in casual speech?" answered "sure, I'm from Tuscany!". I myself fit well in the that local culture.
That said, do I still talk like that when my 4yo child is around? No! I don't want him to talk like that until he can control himself and read the room! Is fixing this bug in the society (the rule against insulting the christian God) a hill I want to die on? Why should I? It's just a word, a fun habit, would it be rational for me to yell at people that I'm free to insult their gods because if freedom of speech or whatever? I'd be looked at as a crazy man because that's what I'd be.
To me people who cling their freedom to utter words that a significant portion of their fellow compatriots find offensive, look exactly like that. Picking a silly battle, and entrenching themselves.