In most environments I encounter, swearing doesn't come across at all, because it's a a natural part of vocabulary that doesn't stand out other than to emphasize frustration occasionally (or also, excitement; what I'm saying is that it's really fucking useful).

If someone where to become annoyed or freaked out because I used the word fuck, or shit in conversation (not in the context of insulting another person; that's a whole different discussion), I would find it very uncomfortable to be around them because I'd be second-guessing my vocabulary constantly. I can be eloquent if I want, but sometimes a "fuck" is justified.