What I learned from Criminology 101 in college was this:

By far, the largest factor of deterrence was the likelihood of getting caught.

So, people, do in fact rationalize "I'm definitely going to get caught and the consequences are bad, I won't do it".

So, to your point, the punishment itself is largely a deterrence. I think because a rational human being simply cannot grok the differences in long term effects. This is already essentially proven in how people behave with money - i.e. people suck at long term saving without external forces nudging them to do so.