How would you handle it? Give them a stern talking-to?
Someone in a sister thread suggested sharia law and chopping off hands, so clearly there are other alternatives besides "society pays for prison" and "the criminal pays for prison".
And yes, I am upset that people steal and that this causes more of my tax dollars to go to heightened policing. Why wouldn't I be? Presumably the only people that are not upset about that are criminals and potentially the police.