You assume that taking offense is always reasonable, or whoever feels offended is always right. Also, playing the emotion card (it hurts my feelings/sensibilities) is a way to excert power and control.
The idea that women have a hot branding iron with them where ever they go that they can use to permanently stamp the word 'sexist' onto whomever they like at the slightest offense is utterly silly - doubly so when it reverses the actual power dynamic that women deal with when in a room full of men.