Unfortunately, I don't think it's as simple as "just go back to smaller groups". For companies, more users = more revenue, there isn't realy much incentive to not try and pull in as many people as possible. And then for most normal users, if they here about some big site, they're inclined to join that so they don't miss out on the the funny tik tok memes/all of their friends being on instagram/etc. For them, what benefit is there to a smaller site where you can't get as many followers or where your friends/favorite celebrity isn't a member.
I agree with you that size is one of the main contributing factors to the problem, I just don't think smaller sites is a practical solution for the public at large. That being said, if you don't care about "fixing" the public problem, then you're right on the money. If you personally don't want to experience toxicity, get off the big sites, it's that easy. I just don't see that being a fix for the average teenager/college student/boomer