I'm in the same boat as you, maybe a little bit less categorical about whether to contribute to projects with CoC. I'm thinking twice before contributing to them, especially for the ones that use it as a showcase, but for sure if once someone start complaining because I made a joke, they won't see me again. For conference I'm even more careful, because of the IRL factor. And when conference use CoC as a showcase, it's a no go. Pycon is on my blacklist for instance.
What I like the most is how they harass people based on a code that said people should not be harassed, because they «do not tolerate harassment of conference participants in any form».
I'm pretty sure I will never be concerned by this because I'm really respectful, but I like jokes in general, about everything. I'm too afraid to become the target of SJWs that have a code that let them harass people because they are not happy with their opinions.