We have come up with words that are easier to swallow like "deplatforming" and "fact checking." Censorship occurs in many forms, and not all of it is directly blocking access to speech (IMO).
There is a common misconception that censorship can only occur at the public (government) level, but not the private level. As per Wikipedia:
"Censorship can be conducted by governments, private institutions, and other controlling bodies."
There is also the issue of orthodox views and orthodox privilege (as per Paul Graham's explanation). Many conservative views are not orthodox to the point where you can be canceled for merely quoting someone else or citing facts (yes, both have happened in the past). So there is some form of self-censorship (this is related to "social cooling" IIRC).
I think it all boils down to this: which side supports censorship and which side condemns it? There you will find your answer on who is more adversely affected by censorship.
Note: this is not a left or right issue, historically speaking. For example, look back to McCarthyism.