Protecting society from criminals (or the violently and severely mentally-ill) is the only function of incarceration that is guaranteed to work. I agree with you that the US justice system is horrific at rehabilitating those who can be rehabilitated - but we don't really have a good understanding of what specific people can and cannot be rehabilitated, or how to go about actually effectively doing the rehabilitation.

Whereas someone who has committed 30 petty thefts and then gets arrested, locked in a cage, and guarded by armed agents of the state, is extremely unlikely to commit another theft as long as he remains locked in the cage.

And he's also extremely unlikely to get shot to death by e.g. a store owner trying to protect his property from theft - another important function of the criminal justice system is protecting criminals from ordinary people using violence against criminals in order to protect their own lives or property.