Tell that to entire towns and villages of unemployed weavers.
Ultimately, the factory owners asked their rich and powerful friends in government to help them out, and they started having show trials and executing anyone who impinged on factory owners' rights to make out like bandits.
If the Luddites made a mistake, it was in breaking the machines when they could've broken the owners. That would make the next set of owners a little more considerate of the human cost of their enrichment.