This doesn't seem true; it is very possible to cause damage with speech alone, anything from hurting someone in a fragile mental state to the point where they might commit suicide, to spreading rumors about someone with full knowledge of what might happen. I'm reminded about the quote from a professor who said he'd rather break his left leg than have a false rumor begin to circulate about how he slept with a student. Your thesis rests on the principle of mind-body dualism (which is increasingly rejected among philosophers and neuroscientists alike), and that speech isn't itself an action (metaphysically, of course it's an action).
>But hurt feelings aren’t something that should ever be addressed by the legal system.
To say that any effect words can have on another person only amount to 'hurt feelings' is inaccurate. Every human alive knows the anguish and pain, sometimes even causing physiological pain, that words can cause. Mental states also have physiological effects of many kinds, such as mental stress leading to inflammation. The principle isn't alien.
>People really need to get thicker skin
Why doesn't the same apply to literal thicker skin or body plating to prevent being stabbed? Is that really what a civil society looks like?