I don't understand how that's possible. When clicking a matrix.to link, there is a room name and a homeserver hostname, right? How does that homeserver not have complete control over what information about the room is sent to the client?
You may be suggesting that matrix.org could, for example, block access to the gentoo.org server (or a subset of its rooms). In that scenario users could choose a new homeserver (and not necessarily the same one) in order to regain access, instead of the entire channel having to migrate.
In the incredibly unlikely event where every homeserver on the internet is blocking gentoo.org, gentoo.org could simply allow user registration.
With IRC you are forced to trust the network operator with both your account and channel. Matrix gives you the option to trust someone else with both, neither, or either.