> what do I do when I fundamentally disagree with someone else about the implicit sexism or bias of an expression?

In this case you ask at least one 3rd party, maybe more. Some of your mentors should be similar to the person who feels victimized, in this case a woman. So do that in real life where you know them, not online.

> What if it's something as innocuous as "low-hanging fruit", and, a colleague is offended by that?

Haha, that's pretty funny and apt. Lead with that when you ask for advice on this.

> I feel victimised, and they feel victimised.

That's also fine. You deserve to have your concerns heard. When someone else is the first to raise a concern with you, it's necessary to listen to them first. Then when they indicate they feel understood (you can ask them directly), you can follow with your concern.