I see your point and it does make sense. But from an IRC perspective, most people/groups only care about a handful of channels. So you might have the #community or #support channel, and someone starts acting up. A channel operator can ban them from that channel directly. Alternatively, a lot of IRC "teams" maintain a bot to manage the channel. The bot can give someone a temporary channel ban (a timeout) based on keywords, or a mod in the channel might type something like "!ban userx because of rule 12" and that bot might ban that user from a list of managed channels.

That user is then still on the server, they can make their own channels or message admins directly. There might be an #ops channel where they can go and plead their case. If they annoy the admins or someone else that reports them to the admins, then they might get a server-wide ban.

A lot of this comes behavior from public irc servers. Channel operators can manage their channel however they desire, and if the users don't like it, they can start their own channel. But for an organization ran IRC server, a server wide ban is more possible. But in the case of say, Mozilla, there are all sorts of different groups running their own channel. So a group working on a mozilla labs project might just start a channel for that project, but not actually have server-ban privileges. So they can moderate their channel and perhaps request a server ban on someone, but that server ban might take hours to approve, or the bad actor might just go away.