Prison itself was an effort at rehabilitation, the theory being that you had plenty of time to think about the wrongdoing. Rehabilitation is ultimately something a person has to want, so you should offer it, but it can't guide sentencing.
The idea that criminals need education is actually pretty racist... the crimes we care about are violent and even young children know perfectly well that violence is wrong.
The real reason for jail is that society is largely self-regulating; it works because we have a critical mass of people doing the right thing with only the most minimal oversight.
So the law-abiding majority needs to see that criminals are punished so they don't feel like suckers for following the law, basically, so they'll continue being law abiding.
We also put people in prison to alleviate the daily terror and misery that career criminals inflict on their neighbors.