We mustn't forget the history that resulted in these racist books. At the same time, it's important to ensure that children aren't exposed to such material.
Removing them from public spaces is a good first step - eventually we'll need to give thought to rummage sales and private libraries (parents often try to pass their erroneous ideas on without the consent of society, of course, and accidents happen).
Given the volume of literature in modern society and the workforce needed to police physical (not just virtual) material, cooperation between all involved interests, public and private, might perhaps be warranted.