1. If everyone says that they will no longer want to work with such a person, especially rich people, who can enforce that want, that person will end up working with poor people. The rich people are making a classist argument.
2. Worse, if that person, cannot even get bad jobs, they will be forced to commit more crimes in order to live. Now, the first crime might have been because they were a bad person, but the second crime is on the people who refuse to work with them.
3. What are other possibilities?
You get out of this conundrum by making prison about rehabilitation. Ensuring that people who come out are changed people who are not inclined to commit more crimes. Then you treat them like normal people. And hopefully, that world, while not perfect, will be better than this world.