One of the fundamental problems with most things on the internet is that there’s unlimited output. A socially dysfunctional shut-in living on the internet is simply going to have a much larger output than someone with a healthy social life that only uses the internet in moderation. Someone who is judicious about their upvotes is going to have a much smaller impact than some who just reflexively upvotes or downvotes everything based on kneejerk reactions. This becomes even more prevalent when you add gatekeepers. Most people don’t have time to be volunteer editors or moderators. Or to park themselves in the new submissions section deciding what should make it to the front page and what should drop off.

So of course anti-social nutcases dominate these spaces. That’s how these spaces are structured.

Well, sort of. The "jerks" thing is true in part, but I think the majority of that feel comes from a disconnect between what people want Stack Overflow to be. Many of the contributors (and the founder) want it to be Wikipedia, a place to enter search terms and find your specific problem and its solution, documented exactly once. But many novices get used to finding answers on Stack Overflow, and they quite reasonably see a bunch of questions and answers and decide it's a place to ask for help, which it a thing that looks similar on the surface but is quite different. And so you get a lot of well-meaning answers/moderators closing the majority of what comes in as repetitive noise, because from their perspective it is, and so the majority of people who ask (quite reasonable) questions, especially novice questions, have a terrible experience initially engaging. But yes, ALSO jerks are there.

Yes. The jerks found meta.stackoverflow.com and for a brief period started a campaign of what is off-topic in each very-specific forum in a effort to drive content to their very specific fiefdoms.

basically after sub-forums like bicycle, gardening etc which didn't fit at all under the programming forum, they opened up for randoms moderators. But the jerks started to abuse it to create a reddit-like web of overlapping forums. in 2018(?) you could not post anything without a question being closed with "offtopic! to ask about this specific setting on android phones with a blue or green protective case you must do so on xyz.stackoverflow.com. closed."

in SO cases they became moderators and/or worked their way into the company in paid positions of authority

It worked for a long while, and things only got bad after a lot of changes from the original rules.

Changes were clearly necessary because SO became too large. So, I'm not sure there was an alternative besides staying small. (Maybe that's the answer.)

No. Stackoverflow is, after Wikipedia, the most sane place on the internet. We have our gripes of course, but there really is no contest here.