So maybe make throwing someone in jail not cost a million.
The biggest thing seems to be preventing those administering the consequences from indulging in sadism, or exploiting those who they administer.
If the consequence for minor property damage(think public defecation, minor vandalism etc) a 2-4 hour shift cleaning the streets, potentially connects people with the consequences of their actions without becoming cruel
Making jail "cheaper for taxpayers" tends to mean things like more mandatory labour, more phone calls that cost $5 / min, etc. Trying to squeeze money out of inmates will only make them resent the system even more.
Make the punishment for shoplifting being tortured to death in a public square, and the cost of prison will go down. Sounds shocking to you? It used to be the standard in the Western world, until just a couple of centuries ago.
Over time, Europe has abolished the death penalty. They have the lowest murder rates in the history of civilization. In the US, there are wide variations between states for violent crime. Unsurprisingly, the states with the most draconian laws have the highest rates of crime.
I've concluded that violent punishment does not reduce violent crime.
Moreover, the violent crime rates for many parts of the US are no higher than in Europe. Coincidentally, those places tend to be the same states that have ended capital punishment.
Now, I'm not drawing a causal relationship between draconian punishment and an increase in crime. I'm just saying there's demonstrably no relationship between "tough on crime" harsh sentencing and a decrease in crime. Fear of violent retribution by the State is not what stops people from committing crimes. We have to look at other reasons.