Agreed, but you know who will push back? The police. The private companies running the prisons. Because it means less leverage, less power, and ultimately, less money.
Private prisons don't exist in many states and represent a very small fraction of people in prison. We lock up so many people because for many years that has been a popular thing to do, and people have affirmatively voted for it. I think those people are wrong, but let's not mislead ourselves about how this happened.
Even public prisons are primarily run and populates through the formiddable lobbying power of police and prison guard unions. Their influence is far more insidiuous than that of the comparably small for-profit prison industry.
Also an opportunity to question your union stance.