Without that kind of protection usually some group gains power and becomes the only protected group. Neo-Nazis are an indication that even the most socially unacceptable groups still have a voice. They should be considered a canary and a good thing to see occasionally. I wouldn't want to live in a country where they were censored and if you feel so strongly about it I would suggest moving somewhere else (Germany censors Nazis pretty aggressively for example.)

For example: You're upset that Nazis marched through a Jewish neighborhood. Are you upset about the violent BLM protestors marching through White neighborhoods? I would guess not but there's little difference between the two.

I've moved to places to see if I thought the changes improved things, you learn a lot about yourself and the world by doing that.