Instead, you should think in terms of a society spending people. Take farming, for instance. We had about 35% of our people working on farms. Then mechanization came to agriculture, and now we have maybe 2% of our people working on farms. What will all the poor farmhands do? Starve? Well, it turns out that farmhands could do a lot more than just farm. But the mechanization of agriculture freed us from wasting an unnecessary 33% of our workers on farms. They went on to do a lot of useful stuff, from which society as a whole benefited.
Don't waste your workers in jobs that aren't necessary. It holds your society back.
Now, the problem comes when you expect that people "deserve" or "need" a job in a factory. That's false, too. They need/deserve a decent job. They may need training to get there. But if you think they need a factory job because they can never be competent to do anything more, I think you are selling people short...