most law enforcement and the justice systems are treated as exactly the type of stuff that doesn’t make sense for a market and should be handled by the government. This should apply to prisons for much of the same reasons.
FWIW, one should also consider that in a Libertarian system FAR fewer people would be in jail in the first place - perhaps as few as none, since Libertarian ideology is not big on forced imprisonment in the first place. The typical Libertarian view of justice is rooted more in the idea of restitution for damages caused (where applicable) than "punishment".