There's also the time component to consider, PC language evolves faster than natural language and a lot of people struggle to keep up. It wasn't too long ago (well within living memory) that negro was the PC term and since then we've been through colored, African American, People of Color and possibly some more I'm missing. It's easy to assume malice where it was never intended and the person was actually trying to use the correct term.
An example I'm more familiar with personally is spastic, my disabled sister went to a school literally called "The Spastic School" until the term was deemed un-PC in the 90's, then it became "The special School" and something else now. I'm aware of the change but a couple of decades on it's not uncommon to run into someone that doesn't realize spastic became a derogatory term, simply because it's not a situation the encounter very often.
The change seems to be even harder for words used everyday. In Australia (and probably the UK) Sheila is often considered a derogatory term for a woman, but many older men will use it the way we use chick/girl/women. It seems like the frequency of use creates a self-reinforcing pattern, you won't find many people under 50 using the word and the over 50's that do will keep using it until the die.
So aside from sensitivity, for many it's just resistance to change for various reasons.