Weird... maybe he was also mad at economists because they proved again and again that slavery was more expensive than paying people a wage!
As for literature backing this up, Adam Smith in Wealth of Nations talks in detail about slavery in the American colonies.
He gives examples where slavery was only viable in the southern states where they grew tobacco. Maybe one of the reasons why the South were so opposed to the end of slavery vs the North where slavery was expensive