The reason that "free speech" is limited is that limiting it does almost no harm, and allowing it can result in human death.
This actually follows well into the specific example of racist nazi propaganda. The argument is fairly similar; allowing that speech has the chance to, in a more roundabout way, result in human oppression and death. Limiting it isn't great (especially since it often draws some amount of attention to it), but it does seem plausible that by stopping that speech from getting out there, there's less chance of such a movement forming.