The same is true in any self-organized community i know of. If anything, it would probably generate an invitation for the complaining neighbors to try and find some better suited place for them to live. That is because the only communities i've visited were very much antiracist, anticapitalist, antisexist. And despite all the instances of racism i've seen with my eyes (we are after all, all chidren of one of the most racist countries on earth), i've never seen in a community the kind of racial segregation and hatred i've seen on the streets of big french cities.
I mean, i could imagine some white supremacists setting up a community. But they would not let the rest of us in in the first place, and certainly wouldn't require a consensus to cause harm (whether psychological or physical) to their enemies. In a barbaric society where prison or death sentence are supposed forms of justice, it can appear that excluding someone from a community is simply a matter of personal preferences or bigotry. However, having to part ways with someone because they're harming your community and there doesn't appear to be ways to resolve the problem, is a serious concern that you may have to deal with sooner or later whether you like it or not. But proposing to exclude someone for a trivial matter would instantly raise red flags as an authoritarian manipulation scheme.
Life in a commune is not comparable to life under a nation-state in most regards. Your neighbors in a commune are not just people who happen to live nearby, they're people you live with. And certainly there's no landlord to deal with who have unreasonable power over others :)