This is incorrect. A considerable amount of opposition to slavery came from "Free Soil" farmers, laborers, and mechanics who did not want to compete with slave labor. After the war they continued to pursue their class interests and supported labor rights and, eventually, unionization.

The northern capitalists who ultimately constituted the financial engine of the Republican Party did turn against labor rights, also in accordance with their class interests (and, depending on the industry they were in pre-war, some of them had been pro-slavery or pro-southern 'Copperheads').