The reason they are so divisive is because the loud people fall into two categories: 1) people who loudly and constantly complain about these issues and for whom no amount of improvement is enough, and 2) people who think all these problems are greatly exaggerated, part of the liberal culture wars, and find it extremely annoying that anyone would bring it up.
The "silent majority" in these issues are people who say "yeah, sure, it seems fair to allow gay marriage, why not" and "let's see what we can do to improve female and minority representation but let's not jump to the conclusion that anyone in particular is to blame".
The hallmarks of being in this silent majority are:
- being generally well-intentioned
- supportive of incremental, common-sense solutions that are not zero-sum
- dislikes intense conflict and blame games
- does not take a view of the issue that makes one group of people the "heroes/victims" and another the "villains"
So, if you hear the proposal "let's allow politics at work", who do you think is actually going to speak up? That's right, the loud and annoying people that you can hear anytime by turning on the news. Ideologues for whom compromise and assuming good faith in your opponents is anathema. No thanks.