Lots of the servers simply block based on tribalism though, irrespective of "toxic content".
My single-user instance, for example, which is entirely free of "toxic content", even with an extremely critical viewpoint, is blocked by an admin of a popular public instance of 17k users, none of whom can follow me (or can I follow), simply because I publicly disagreed with him about the virtue of instance censorship.
These things tend to naturally centralize, and that's always going to enable small groups to have an invisible, unearned, outsized influence over what people can see or read.
I think client-server is the wrong model for p2p networks. Look at the issues with the deliverability cartel and the relative inability of normal people to deliver messages to Gmail users without paying money into it.