They absolutely could be. A cooperative community full of racists might very well attempt to kick out people of certain races.
> you have a status quo where other forces can also banish a person.
Ah, see, but we don't have that status quo. Nobody can kick me out of my apartment, as long as I keep paying rent. And that paying rent part, isn't even strictly something that would get me kicked out, in certain places!
There are tons of laws and rules and regulations that very much prevent people from being kicked out of places.
> i don't get what you're getting at.
What he was getting at, is that it should be obvious that just because a majority of people want someone kicked out, that this might not be a good reason to actually kick them out!
EX: if a majority want to kick out certain races, we, as society, should prevent that majority from democratically excluding certain races.
So, in other words the answer to the question of "why should you be allowed to live somewhere if the community wants you out?" is because we live in a society, and in that society we want to prevent racists from kicking people out of where they live, even if they kick them out democratically.