If you have something worthy of being said, you should use that to build an audience of people who bother to follow you, so when you say something controversial you have people who will hear it... and we should be building a system that ensures that that can happen without interference. You shouldn't just get to reply to someone else and shove stuff at their audience, if nothing else because that doesn't scale.
In my case, and to show that I have some "skin in the game" here: I happen to have a rather large following of people who listen to my let's-call-them-"rants" about Apple (and sometimes Google and Facebook, but my audience mostly cares about Apple), a company with much larger resources than I do... because I have something valuable to say on the subject and have built my audience up by being a trustworthy and even unique source of information.
But, I shouldn't get to force my rants about Apple onto Apple's audience of people who don't know I exist and who only wanted to sign up to listen to Tim Cook or talk to Apple's corporate support or communication lines... that's just me clogging up their communication channels. People should get to opt-in to my content no matter what I want to say, but that doesn't mean I should get to force what I say on anyone.
Put another way: the problem with not doing this is that you get what I guess Twitter kind of is (and it sucks): a place where the people who shout the loudest and post the most message volume--almost certainly because they have too much time on their hands from having nothing else valuable to do with their lives--manage to get the most "space" on the platform by just going around replying to people who are trying to provide high-value content.
This current situation, I will then also argue, is much worse for the goal of the little guy/gal with the niche opinion: it implies that the platform and the zeitgeist get to co-determine what is shit/spam via some horrible combination of a centralized (and clearly biased) overseer reacting to a "tyranny of the majority" (or sometimes an authoritarian government), which results in a world where you don't even get to talk to your own followers, much less mine.
This much-more-serious censorship--with people losing their accounts or their tweets getting filtered by centralized moderation based on keywords in an attempt to scale this process across all accounts at once--is a downstream effect of attempting to almost provide a "right to be heard", because we do generally want--nay: need--the ability to filter shit/spam from the platform: but to scale and be fair it needs to be done at the level of individual users and their own audiences, not with centralized filters for the entire community.