Yes human are moral agents and at the same time every human are different. What one called "wrong" might not be wrong for other.
Easily: what is right improves the human condition and reduces suffering. What is wrong does the opposite.
Let me short-circuit your next question, which is surely "how do you measure what improves the human condition, and what is suffering?" We have to use our judgment, which we are graced with by virtue of being moral agents. This is subjective, like anything worth doing, which means we sometimes get it wrong. And that's fine. The fact that we sometimes imprison innocent people doesn't invalidate wholesale the concept of a system of justice.
If you don't allow them to be part of the world you end up alienating them forever and they can never redeem themselves and change.
Society has forgotten how to forgive.