I agree selective freedom of speech is hard. Probably too hard.
I tend to support selective augmentation of other people's speech - "fact checkers have shown this to be inaccurate" next to a conspiracy fan's latest comment. As long as it is more precise than just downvoting to oblivion.