We have always understood that there are consequences of unpopular speech that are detached from whether that speech is good or necessary or correct. It's why we characterize people as courageous who fight long, but ultimately justified or righteous battles in the popular sphere. Courageous in the face of what? This! The negative social consequences of unpopular speech.
It's a cute term for freedom of association, which is a right even more fundamental than freedom of speech.
No justice more savage than a bunch of people on Twitter saying mean things about you
True, just saying mean things isn't savage justice. Savage justice is when the lynch mob successfully bullies your employer into firing you.
After someone get fired from their job for saying something evil, they still have the freedom of speech to continue saying it.
Savage justice is when somebody comes to your house and beats you to death with a tire iron. People calling JK Rowling a hateful bigot on Twitter and telling people to stop purchasing Harry Potter content isn't that.
Why do you hate the freedom of speech? It seems odd that "free speech absolutists" only seem to care about the speech of right wing trolls and even want to curtail other people's freedom of speech and association in doing so.