It's not being hostile, it's more than that. For example, if I invite my men co-workers to go swimming then there is absolutely nothing wrong with that. But if I invite my women co-workers, then it could be interpreted in a bad way.