It's sexist regardless if woman are present. People are victims even if they stand their ground.
By this logic, you have to ban humor.
There is situation in which everyone mocks everyone. But the situation described here is that some people are valid target of jokes and others simply dont.
There's plenty of jokes about man-bun and skinny jean wearing hipsters floating around. There's plenty of jokes about white trash floating around. I'm not sure what a joke about white dudes in general is going to look like barring the obvious case where you generalize a joke about white trash or hipsters to all of them. It's going to be very hard for a group of mostly white dudes to joke effectivly about white dudes because as members of that group they all see that group in higher resolution than that.
I am currently at an hotel with an international audience and was part of the daily 'show' a couple days ago. I made the entire audience laugh when I joked about not knowing how to sexually satisfy a woman (I am a white man if it helps).
Think it depends a lot on the general tone. Are women targeted for sexist jokes as much as men and the jokes are free for all? Probably ok. If 95% of the sexist jokes are about women, probably not ok.
But for example, I found that a lot of Eastern European men make sexists jokes that are mean and at the same time, can't take the smallest joke about themselves (or worse, their own sexuality).
Context and culture matters.
Go to a comedy show and listen to how many black guys make fun of white vs blacks. Go to another one and listen how there are lots of jokes about husband vs wife, men vs women etc.
The problem is that there is a time and a place for those and a professional environment is simply not it. This is why I always have a problem with American culture of after work meetups and socialization.