I also grew up using pansy as in weakling, as in like the flower. Gay men can be pansies or not. Straight men can be pansies or not. Sometimes I am a pansy. Sometimes I am not.

There is a connection to offensive meanings even if it's not strictly a slur. It's the wrong word here anyway--really the opposite of what I would take as the intended meaning. The behavior being argued here certainly isn't weak; those arguing against it are saying that it's overly aggressive.