If, instead, you mean, civil rights for members of Muslim extremist organizations, then your comparison would be reasonable. So, you tell me. Should Al Qaeda be given permits for rallies across the US to recruit people to commit acts of terror?
We aren't talking about people who merely say racist things. No one is suggesting racists should have their speech curtailed, as repugnant as that speech may be. We are discussing groups that have as their stated goal the violent overthrow of the US government and imposition of a white ethno-state. It's not a secret that's what they stand for and what they are recruiting for. They talk about it pretty openly, and it can be confirmed by spending even a little time in their online or real world spaces.
It is false equivalence, and gross, to compare Neo-Nazis to Muslims. There is no similarity there. There are violent extremists that are Muslims, but it is not the stated purpose of the religion or the majority of the people (or even a notable percentage) who practice that religion. Neo-Nazis, the groups represented by the Unite The Right rally, are violent extremists, both in word and deed, and it isn't a secret or small percentage of their numbers. It is why the organizations exist.