I'm not even against the telling of jokes that are. . off-color. But they really don't belong where you might cause distress. If you think something is offensive enough to warrant looking around before saying it, maybe save it for home.
My girlfriend (a Jew) and I crack Jew jokes all the time. . .at home. Even though no one in my workplace is Jewish (that I know of), I still wouldn't crack most of the jokes we use at home in my workplace. It's just not appropriate, not by any reasonable standard.