If it were, it doesn't take a bodybuilder to kill or hurt someone with an axe or a knife.
But that's completely besides the point. The point is that in marital abuse - getting hurt (physiologically or psychologically) by someone that was supposed to care for you is always demeaning and hurtful experience.
And its a point that few women know, that male psyche is way more fragile than male physis. Some know and take advantage of it.
Your statement of men not being hurt by verbal (or other forms of psychological) abuse is frankly, naive. A man puts up with it because, frankly unless he moves away (and that's what every sane person should do in my opinion) there is no way for him to defend himself from his mother/spouse without getting a short end.