(In normal workplace setting women are actually shielded from that friendly fire because men don't feel as comfortable/can't predict what's OK and what's not)They didn't used to be shielded like this, because men didn't used to think "what women thought was OK and what's not" mattered. In the same way, "white" men used to feel universally entitled to make off-color comments about the silliness and inferiority of "non-whites". Now the power relationship has changed, and now there is some thought given to what's appropriate and what's not.
You want jokes, you have to acquire certain skin thickness. If you don't, then I don't want to work with you. Fun is fun.
"Skin thinness" has to do with a certain degree of wounding and the remnants of institutionalized injustice. Both the wounding and the echoes of the injustice in the culture might take some generations to completely fade away. That said, all people have fun. All people want to relax and be genuinely themselves with someone, sometime. Are you entitled to have fun with absolutely everyone in the particular way that you know? I'm sorry, but it might not even go as far as the small subset of your workplace.
"Fun is fun" is pretty naive. If one studies humor, one finds that it's mostly highly contextual and cultural. This is particularly true for what's acceptable between men and women. That's not some conspiracy to rain on your parade, it's just how things are in the world. Just get to know people first.