Why would that be?
If A harms B, then your argument is that B has no right to retaliate it, because, that A should harm B was inevitable, and was totally out of the control of A.
But my argument is that just like A's harming B was inevitable, so is B's judgement of A and the further retaliation in response to the A's action..Inevitable.
In other words, the world should just go on as it is, even in the total absence of free will. What is important, I think, like many other things, is the 'appearance' of having free will. Because without that, all life looses it's meaning.