Who gets to decide what ostracization is good and what ostracization is bad?
No they don't, there are laws against that, for example you can't decide to ostracize black people from your company, you will get sued for that.
So its fine to fire all black people to ostracize them? No, of course not, so no you are wrong here, there are more limitations than that, people have rights protecting them from a lot of ostracism.
This works because people can choose to not be jerks, but they can't choose to not be black.
What if they could? People can change gender identity nowadays, and it's not outside the realm of possibility that people could change their phenotypical characteristics (transracialism is a hotly debated topic).