Oooor, try this one on for size:
What if they're not out to cause offense and the malice you impute is just an illusion under which you yourself are laboring alone? What if it was a well understood and not particularly offensive vernacular usage from before people decided they ought to spend their time being offended on behalf of hypothetical listeners?
They were stung. "I'm not poor!" I felt so bad about it that it's stuck with me all these years. Does that mean because I've seen first hand how hurtful it can be that I should chide people whenever they use the P word?
And that analogy isn't even accurate because I'm not the one informing you that the word can be hurtful. You're using it already knowing that. So a better question is did you continue to say "oh, poor baby" to the kids who were hurt by your original comment?
I don't think you would continue using it, that was the point I was making and it sounds like you now agree with me that we shouldn't be knowingly offensive.
Yet in the years since, I still talk about poor decisionmaking, poor luck, poor performance, and poor word choice. Because it would be poor logic to go through life auditing everything I say just in case a middle schooler with a somewhat poor vocabulary might mistake my meaning.
"Poor" has non-offensive uses so you can continue to use it in other ways. You don't need to advocate for the non-offensive uses of a slur. You know regardless of the context, some people will be offended by its use.
In fact, while I wasn't around at the time I'd wager that "mentally retarded" came into an official usage specifically because it was a clinical, sterile, bloodless, and utterly anodyne descriptive term. Moron, imbecile, and idiot all were once clinical terms. And then people throw them back and forth at each other to call each other stupid, they gained an offensive connotation and new terms were needed.
In 20 years will you find it absurd if people say that "differently abled" is a slur? Will you say "this is nuts, we literally came up with that term to avoid offense?" I will!
More seriously. Is "moronic" okay? It's just an ever so slightly more archaiac way to say retarded. The meaning, the negative connotation, the level of offense—it's synonymous and analogous across the board in its time.
Is it okay because "nobody means it like that, they're just using it as a synonym for stupid?" If so, congratulations, you now understand the other side of the argument better than when you started.
It is a fairly common conflict that arises as a flashpoint in many areas. Different social and legal theories come up with radically different standards.
If someone has a cold, should they not go shopping out of caution for others? If someone is immune compromised, is it their responsibility to take precautions in a store?