This is an age-old refrain and I am honestly surprised that people keep viewing it as a reasonable position. It's exactly the same line that Parler tried to peddle, along with every online community that eventually - shock horror - turns out to be a platform for hate rather than 'rational' thought.
Free speech is important, and should be defended. When your version of 'free speech' is only extended to alt-right ideals, you're not arguing in good faith for free speech.