Popper's Paradox of Tolerance [1] is a more direct expression of this:
> The paradox of tolerance states that if a society is tolerant without limit, its ability to be tolerant is eventually seized or destroyed by the intolerant. Karl Popper described it as the seemingly self-contradictory idea that in order to maintain a tolerant society, the society must retain the right to be intolerant of intolerance.
Sometimes, that intolerance of intolerance can be sufficiently dealt with by society merely expressing disagreement, but if the intolerance expressed involves direct abuse and threats, for example, by people undeterred by their opponent merely countering their speech with speech, the only non-violent means of preventing the harm, including limiting the liberty of the victims, may be through the application of legal restrictions.