Why is it not part of the punishment?
I think it is dependent upon interpretation. Otherwise arresting and locking up can be interpreted as kidnapping and so on.
I do agree that they should be fairly compensated though.
Indeed it is part of the punishment. As an earlier comment in this thread points out from the 13th amendment:
> "Neither slavery nor involuntary servitude, except as a punishment for crime" (emphasis added by me)
They're not. That's why it's slavery. And slavery of prisoners is permitted by the Constitution, which is the main point here.