I don't think this is correct, abolitionism was a thing particularly within the most religious (read:moral) groups of people. Only a tiny fraction of southerners owned slaves, most estimates put that number at under 2% of the population, and there isn't much evidence that the owners of slaves even claimed to have the moral high ground, let alone the 98% of the population that didn't own slaves.
That's not to say abolitionists saw all races as equal, and it also doesn't ignore those who tried to roll out the story of Ham to justify slavery, but there was no consensus on the morality of the institution even at its height. If there were and if it were considered the morally "right" thing to do then we would have seen far more people becoming slave owners, especially among the most religious.
Slavery was essentially a business decision, not a moral one. And not a very good business decision either, even if you can look past the evil of it all.