Yes, trolls are a problem when one wants to promote free speech. It means you cannot censor them, but you should find a mechanism to discourage such destructive behaviour. Maybe a similar system that exists in real life and real personal relations. Maybe some form of hierachy. Unfortunately I do not know. However, I would look into establishing a meritocratic system or a representative system (much like the political system), where like-minded groups promote their representatives that have greater reach. And the anti social behaviour is punished at the social level and not by central purging.
Regarding your comment about ditching free speech and allowing only what I think is good. It would imply that I know what is good. I do not have such perfect knowledge and I do not believe anyone else has.