Of course intent doesn't change the harm done to victim, but I believe it definitely changes the way we should look at the perpetrator.
In a perfect world, the only thing that would matter in a justice system would be intent - it would determine whether one's to be punished or left alone. The issue of harm to the victim could be covered by insurance. But such a system would require something that could determine intent and rely it truthfully. We don't have such an instrument, so we're stuck with a system that considers everything but intent in order not to get gamed too easily.
Again, what I think we need to figure out is not how to just limit harm done to the victim (which, of course, is not dependent on intent), but how to push back against the perpetrators who are clearly acting in bad faith but stay on the legal side. Just like we don't only instruct people to keep their valuables safe - we also go and catch thieves.