Sure, maybe if you're looking for reasons to take offense, "just" is one you could take offense to. But ultimately I think if we all are walking around on eggshells to the point we can't say that word, that's a much worse environment than one in which people say "just".

Besides, just because you take away the word doesn't mean they can't say it. Removing the word just makes it harder to recognize when people do it.