I disagree. Never in my life has the best solution to anything been the extreme, instead it's always been balanced and nuanced solution that have ended being more optimal. I've observed this in science, physics, engineering, medicine, culinary, inter-social relations, business, in everything.
Based just on that, I already get a red flag when someone says that an extreme free speech position, such as free speech absolutists, is the best solution to have a liberal society with civil liberties.
In my experience, balanced and nuanced is always better, and you must always have provisions to handle special cases, there will always be special cases.
Free speech is very important, like you said, beyond violence, how else you going to influence and assert any improvement or changes to defend, retain or ask for more liberties?
But speech can remain civil and respectful, it doesn't have to include slander, or threats to a person, insults, ridicule, raised voices, talking over people, etc.
And then you have the conflict with other liberties. Yes governement shouldn't be able to restrict anyone's speech, no matter what. But between themselves, citizens should be allowed recourse to slander for example. They should have ways to protect their reputation when it is being harmed by someone intending to harm it through misinformation, lies, and false accusations, and that would require the government to intervene and uphold someone's right to their reputation.
Also, we should be allowed to create communities with code of conducts that include speech behavior. If you come to my house and start bad-mouthing my daughter, insulting her, and I kick you out, you shouldn't be able to sue me for violating your free speech rights. I should be allowed to control the code of conduct of others in my own home. Similarly on my website, in my comment section, or if I am a company running a social community space I too should be allowed to do so in order to protect my business. Robo-calls, spammers, calling me or leaving spam in my email, website, I should be allowed to block them, or have some service that auto-censors them, it shouldn't be that their creator can prevent me from blocking them because it infringes on their creator's right to free speech. Same for bot accounts and all that. So clearly I feel there needs to be special provisions for exceptions to the right to free speech, and when exactly it applies, to whom, about what, and how the speech itself is conducted and what the speech is actually saying, is it false accusations, slander, threats, spam, etc.