I 100% agree with you
in principle, but I honestly never thought we'd be having this debate in 2017 about people carrying swastika flags and calling themselves Nazis. Frankly, that changes the entire equation for me.
We cannot, cannot, cannot allow this to take root and grow. Everything thinks "it can't happen here", but it doesn't not-happen on its own. I don't want my grandkids asking "how did you try to stop them?", and having to answer "well, I very adamantly told them they were wrong but they didn't listen."
I am incredibly tolerant of diverse idea. I grew up in the buckle of the Bible Belt, and now I live in the bluest of blue places. I have wonderful lifelong friends in both places (and lots of others) who I love as people but whose opinions infuriate me. That's OK! We talk about stuff and occasionally ruffle each other's feathers, but we get along just fine. I can argue both sides of gun control, abortion, and anarchism with clever friends for the fun of it. But what I can't tolerate is people in my America wearing, carrying, and acting out Nazi symbols in my backyard. That cannot be allowed to find fertile soil.
I've said this before but I want to be perfectly clear: I have an exceedingly tight definition of that term. I have never called someone that word because I disagree with them, no matter how important the matter to me or how ferociously I oppose their opinion. I'm referring specifically to people who show up at white power protests wearing the uniforms, carrying the flags, and throwing the salutes of the group my family fought to stop within my mom's lifetime.
That's not a slippery slope. It's perfectly possible - and reasonable, and good - to say "we'll allow people to speak abhorrent opinions" and still draw the line at literal Nazis.