I don't care about what murderers or white supremacists think. Their morality is wrong, and they don't deserve a seat at society's table.
If you don't allow them to be part of the world you end up alienating them forever and they can never redeem themselves and change.
Society has forgotten how to forgive.
Easily: what is right improves the human condition and reduces suffering. What is wrong does the opposite.
Let me short-circuit your next question, which is surely "how do you measure what improves the human condition, and what is suffering?" We have to use our judgment, which we are graced with by virtue of being moral agents. This is subjective, like anything worth doing, which means we sometimes get it wrong. And that's fine. The fact that we sometimes imprison innocent people doesn't invalidate wholesale the concept of a system of justice.