Blue colar workers did not profited from slavery. The interests of free labor were one of reasons for slavery opposition. Blue colar worker can't compete with slave (or ther slaveholder using slave).
https://www.commentarymagazine.com/articles/labor-unions-and...
(note the date)
I wanted to disagree with that one specific point.
There was much less of it in states where slavery was legal.