It's more like "You're free to believe whatever you want, but that freedom doesn't automatically make you not a bigot".So many words for 'we won't accept it'! I'm not saying that all beliefs are equally valid. I don't even have a problem with calling people bigots, as long as you're careful to mind the gap between 'speaking' and 'acting'. I am saying that you are not the only one who really, truly, actually believes that they're running out of patience and that 'doing something' is justified.
It doesn't matter how strongly you believe that you're in the right. If you decide that, just this once, it isn't really imposing your beliefs on others then you will have to live the rest of your life in a world where everyone else decides that way too.
(Not to say that you won't have to live that way even if you don't do it. Other people's beliefs will impact you, more or less by definition. On the other hand, it doesn't make for a very convincing justification.)