You can look at the length of a sentence as corresponding roughly to a cost benefit analysis pertaining to the given crime._You_ may be able to. But most people who commit crimes, don't. They're experiencing brief or not-so-brief lapses in their planning capability, and they're not looking more than five minutes into the future; Extra sentencing is useless as a deterrent to these people, because they're resistant to any consideration of sentencing. They're just not looking that far ahead. Organized crime is the exception to this rule.
Notably, they're not immune to the effects of a high arrest rate. If someone has a 90% chance of 1 year in jail, it is an inordinately more powerful deterrent than a 10% chance of 10 years in jail.
The cost being the cost to incarcerate the person, and the benefit being the person's inability to commit a similar crime for X number of years. So for really bad crimes that carry a really long sentence, the benefit to society of there not being a potential recurrence of the criminal's acts outweighs the cost of the incarceration.
This is a different consideration. This is "incapacitation" rather than "deterrence". It's explicitly giving up on the ability of a prisoner to make decisions, either before the crime or after the sentence; A criminal is thus destined to commit crimes, and 'taking them off the street' is the goal. That's basically how we've approached this for the last ~40 years of progressively expanding incarceration rates, unlike most of the other countries around the world. And that's a fairly inhumane, morally questionable view to take, not least the end of the racial persecution of minorities for being minorities dovetails precisely with the racial persecution of minorities for being criminal in some way, living in a place with high criminality or being tangentially involved in the drug war. Broken windows policing, where crimes may be invented, or activities criminalized, in order to get "at-risk" individuals into the system, where they are never permitted to fully leave.
We talk about lynching as some sort of widespread historical horror, but it only happened to 5000 or so individuals. There are over a million black men in prison, and a lot more than 5000 dead as a result of things like the drug war and racialized pushes for ever-expanding sentencing.
Why do I bring race into it? Because repeated waves of contentious race relations are the big sociological delta between our country, and most of the rest of the developed world. And overt or not, it's been central to how we've done any number of things differently, from the suburbs to the prisons to welfare to the healthcare system to our present political dysfunction.