Society should pay. Society has a responsibility to treat everyone - even the lowest of the low - with basic dignity. Society has a responsibility to reintegrate and readjust those it has failed.
>Society has a responsibility to reintegrate and readjust those it has failed
Where is the accountability in your world? If I pistol whip somebody in the face and take their wallet, then it sounds like (by your logic) that society has failed that person, rather than that person failed to abide by the shared social contract (to not engage in criminal behavior).
How many are in prison for non-violent actions that harm none but themselves?
When we create this nebulous idea of a patriarchal "society" that should guide us in all things, it is easy to blame all of one's shortcomings on society.
"Hey, I didn't get into that college!" "Society has failed you, sorry."
"Hey, I didn't win that track meet!" "Sorry, society will do it's best to not let you down in the future."
I think that if you ever try to cash in on what society owes you, you will find yourself woefully disappointed.
(perhaps not backbreaking slave labor, but stamping license plates doesn't seem cruel and unusual)
The problem is that the work they done is used to replace standard jobs and to make a decent profit for those in the chain, which creates a perverse incentive where you now have those who wish for more prisoners as they will make more money. We already have a problem with overcriminalization of actions.
There has to be some incentive for people to go through the trouble to use prisoner labor. If you take away that incentive there won't be any work for them, which seems like a loss if the ability to work for modest pay is good for them.
In general you would hope a handful of employers who like using prisoner labor wouldn't have so much sway in the country as to single-handedly crank up the incarcerated population for their own gain.