There are many good reasons why one might want to refer to a person's biological sex in conversation.
For example:
"Why is that man being allowed to compete in a women's sporting event?"
"Why is that man flashing his penis in the women-only area of the nude spa?"
"Why is that man calling himself a lesbian and haranguing actual lesbians for not being sexually interested in him?"
And so on.