It's like when you are a member of some club, let's say a wholesale club. If you are a member, you are allowed to enter and follow the rules of the club. If you are not a member, you do not get to do that and must leave.
You are free to create your own wholesale club with your own rules, but conflict arises when you demand or even force entry into our club illegally. That is where we have been for many decades now and security is enforcing the rules because the members of the club that have been paying are getting extremely pissed off after having had to pay for freeloaders for decades now and be abused and disrespected by them for it on top of it.
It's not hard to understand, it just seems most people are intentionally and deceptively feigning ignorance. If they are not, we have a way bigger problem that will invariably result in open conflict at some point.
If you want to free speech, I suggest you start your own free speech club wherever you are, or if you gain entry to our free speech club, you have to follow the membership rules and if it were up to me, you would have a trial period, restricted rights, and you would have to pay a freeloader rate.
These are all very reasonable things to abide by in order to keep peace, but we are now all having to learn the hard lessons of what happens when you allow bad behavior to get away with things, and you do not enforce clearly defined rules against those who even blatantly defy them and disrespect you and what you have created, and even do so in your own home. It didn't need to be like this, but it seems most humans are incapable of simply letting well enough be. It seems one of the most universal human characteristics. It applies from the personal to the government to the ideological/psychological level and it is the origin of all human conflict; you have something someone else wants and they'll end up stewing over it until they try to take it from you by hook or crook if you allow them to. The peace comes from keeping a balance. There has been no balance for as long as anyone is old and then some.