Hint: women, in particular, live in almost constant fear of their lives and their personal safety, in a way that most men do not. They simply do not know who among the men they interact with might try to rape them, possibly until it happens.
As social beings, one of the things we do is subject our fellow humans to subtle tests of their trustworthiness. If they show signs of empathy, that increases our faith that they will not seek to actively harm us.
Refusal to submit to the test is failure. It is interpreted, at a subconscious level, as meaning you have something to hide. So "playing the social game" becomes an essential business skill. Sperglords also have a great deal of difficulty with empathy, so it shouldn't come as a surprise that they are shunned by society at large, especially shunned by women, and considered toxic in associations that hope to be supportive of female members.