Ok so I get that you disagree but how is it violence?
Usually, this is taken care of through the courts, wherein the judicial system punishes people for crimes -- again, an act of violence. However, in the absence of punishment or a judiciary aligned with the interests of oppressors, the oppressed are morally justified in enacting punishment and responding to violence in kind.
[1] Paulo Freire, Pedagogy of the Oppressed, 1967.
You might want to ask a black person or an Indian for their perspective on this.