Now with that said freedom of speech is a special law. And obviously one that cuts both ways. But a key reason why the law should largely exist untampered is that even if you say something I think is offensive, I can rely on a few things:
1) If it's absurd it will be discarded by most people.
2) If it's absurd those people it would convince, probably already believe it.
3) With freedom of speech I can argue the point prior to irreversible damage.
Now there are some cases where freedom of speech don't apply and those are generally cases where the three points above don't apply. For example, the classic of yelling "Fire" in a crowded room. Someone else yelling, "there is no fire" is not likely to be heard and even if "Fire" is seemingly absurd, most people may not discard due to potential harm. Yada yada.
But outside of cases that violate the above, the law should stand. Otherwise the consequences, shutting down opposing speech, will almost certainly be worse.