According to https://en.wikipedia.org/wiki/Black-and-white_dualism this is traceable to the Ancient Near East and listed in the Pythagorean Table of Opposites for example, so it goes much further back than the Atlantic slave trade.
I do not think that it would be possible to rip this metaphor out of our thinking.
By the way, a really great linguistic book about how much of our thinking is in systematic metaphors in general, is "Metaphors We Live By", by George Lakoff and Mark Johnson.
But this is exactly what certain people are trying to do. And it's not a fringe group anymore, this is starting to touch different aspects of our life that have no any real relation to racism or skin color. If "black" in "blacklist" is offending you today, tomorrow you will be offended by Master's degree.
Changing these words is a feel good thing, that does nothing to address our internal biases, prejudices, and racism. It wastes time that could be spend on writing your senator, politician, or being active in a political party.
It's a zero effort, zero stakes thing, that makes privileged people feel personal accomplishment, while allowing them to relax into complacency.
"I make a million a year, and work for a company that makes money off of algorithmically radicalising people into right wing conspiracy theories, but I don't use the word white- and black-list anymore, because it reminds me of race. I did my part!"
black is bad everywhere, probably because of its physical properties. ditto for white.
Those who wish to see a racial relation are just racists, and racists should not be fed.