If the instance admin becomes abusive, or treats the server like his own petty little fiefdom, what can be done about it? Why should I trust an instance admin?
If I start acting like a jerk, them my users will start leaving. Pretty soon I don't have anyone left except people who like being treated like that. In my experience that will probably mean they're jerks too.
And thus pretty soon my instance is "that instance run by that jerk who lets her users be jerks". At that point, admins on other instances stop federating with my instance. Eventually the only other instances who are willing to talk to mine are also run by and full of jerks, and I am amidst a hell of my own creation.
So that is, as I see it, the external forces acting on me to try and be a chill admin who's nice to her users. It's a lot easier for them to leave my server and stay in contact with people elsewhere than it is to leave Twitter.
Moving instances is not a big deal and yes, know your admin.
Instead most "blocks" are just muting from the federated timeline, precisely so that you're not cut off from contacts.