The justice system punishes society by failing to adequately rehabilitate offenders. As a result many go on to reoffend. So really the current system is only punishing the next innocent victims of crimes perpetrated by released inmates.
See the difference in reoffending rates between Norway and the USA, and their different approaches to prison time. For a start, the US views the inmates as "criminals", Norway views the inmates as people who did something illegal.