Part of the problem is that even thought a lot of people generally agree that we are too harsh on criminals, they don't stick to that viewpoint when the rubber meets the road during situations they care about.
People who are liberal on sentencing get angry if short sentences are handed down in rape or financial crime cases.