Freedom to speak ones thoughts without fear of a mob attack should be absolute. Anyone who seeks to silence the speech of others is only after power, not peace.
Freedom of speech is an important buffer that prevents violence, too. Far too often the group calling to silence some unpopular opinion doesn't realize that in taking that outlet away, all that is left is violence.
The reality of a free society is that we will be confronted with ideas that differ from our own. It will often be uncomfortable. To be free is a demanding thing for a sovereign citizen.
The exact thing to not do is to claim that every situation is potentially worthy of universal condemnation and that we must figure out how every situation could potentially be worthy of universal condemnation. That's the opposite of living in a free society.