Gee imagine being fired or ostracized for voicing casually x-ist comments even though they were true...
Imagine being fired or ostracized because of your sex/skin/passport though... stop whining, start asking questions. What does the experience of being x feel like at your company? Just ask and listen.