We know Africa and South America and Southeast Asia are still relatively dangerous. That doesn't excuse the sentiments that are building here in the US whatsoever. It actually doesn't factor at all into this discussion because they are totally off the radar for people who should be provided the liberty of living without constant breathing down their neck and "go home" and "jews will not replace us." Where does the liberty from harassment and execution factor into your liberty ethics?
The issue is also not what happens right this moment, but what happens in 10 years' time when people have been surrounding themselves and their family and friends with this rhetoric. The history of the Nazi party following the fall of the Weimar republic doesn't span months, it spans a decade or more. What reason is there to deny some people the "right" to demonize and alienate minorities solely on the basis of inaccurate judgments when things could get a lot worse down the road? The slippery slope, after all, has two faces. We can just as easily say that granting some liberties now causes a lot of hurt down the road as removing them.