Evidently, you are in fact not "the semantics guy".
By whose definition?
The World Health Organization includes not just actual but also threatened physical force or power that results in injury, death or psychological harm
Something that causes harm is harmful; something that causes hurt is hurtful.
So something that is harmful or hurtful is violence?
Or what am I missing?
Eh, when I was a kid violence was physical by definition. That distinction seems to be getting lost such that the phrase "physical violence" is no longer redundant. But the new usage is figurative.