You are not allowed to make protected groups feel harassed or discriminated against. Women are a protected group.
Its a federal workplace law. A company will correctly fire you if you cause women to feel harassed or discriminated. Peoples understanding of whats allowable needs to catch up with the implications of the laws.