"That is in its own way bad for free speech; what use is speech if nobody is listening?"
I think if you carefully sit down and work out what "free speech" really means, it also comes to encompass the right to not have to listen to things you don't want to listen to, via avoidance. (Not via taking it all down, and there are some intricacies I'm skipping over because this is an HN comment and not a blogspew; for instance in "public spaces" there is a certain element of not necessarily being able to just "turn off" someone on a soap box, but still.) This manifests because of the way some people will insist that you must listen to them and that if their particular content doesn't show up in some feed somewhere, they're being censored.
"Harassment can genuinely make the internet or parts of it unusable for some people. "
True. However, the "boil the ocean" solution that everyone seems to be reaching for of some form of eliminating all the bad behavior on the internet is not feasible, even before we get into the fundamental ambiguity of what "bad behavior" is, and the IMHO observable fact that once you start down this road you get the twin problems of A: people deliberately skewing the definition of "bad behavior" in their favor and B: people deliberately getting more and more theatrically sensitive to "bad behavior" in proportion to its decreasing presence, for several reasons.
I think what we're really looking at here is increasingly undeniable evidence that this form of community does not scale. Twitter is fundamentally unscalable. Tiny little Gab may well have already passed its community scaling event horizon. Facebook shards the world much more carefully and still probably is past its scaling event horizon, too. Reddit shards the world in a different way, but its mega-subreddits also often pass the community event horizon. Basically, I'd say that any company it is whose ambition it is to create "one global community", even one broken up like Facebook's or Reddit's, is doomed to failure at scale.
(HN itself may already be on the large side of what can be sustained. It has, I believe, two full time moderators and curators of the community, so when you look at HN you have to keep in mind that that's where this site is; already at "full time moderator" status. If you put up a new "free speech!" HN-2 and promise not to moderate anything, what you see here on the real HN is probably literally unattainable to you.)
Given the choice between Free Speech and Facebook/Twitter/Gab/etc., I'll take Free Speech. We're going to have to figure out some other mechanism for online communication that doesn't involve trying to pour every single person who speaks a given language into one small room. Many of the people reading this probably have personal experience as to how bad that idea can be if it just includes their direct family or circle of friends, let alone millions of people.
(I've actually believed this sort of thing for a long time, and I live it. HN is probably the largest community I actually participate in. I participate in several subreddits, but pretty much only ones that are cut off from the "main body" of Reddit entirely. Even my blog has been affected; I know what it would have taken to "be successful" in that space over a decade ago, and I found myself reluctant to put the work in to do that, just because I realized I don't actually want to build a community and be responsible for it.)