I took it as labor people - those who do manual work for compensation. Field, work, carpenter, etc. It did existed during that time period in states where slavery was not legal.
There was much less of it in states where slavery was legal.
Blue collar workers in the North definitely benefited from slavery in the form of lower cost of living as many items were imported from the South to the North. Also cheaper imports for northern manufacturing increased the value of norther labor.