Why would that bother you any more than that the networks you're on can see everything you type/read on IRC? Generally people are on 2-3 networks at most, and you could get a very good idea of someone from intercepting messages on any one of them...
Networks, by and large, do not intercept messages like this. The only reason someone running an IRC network would see your messages is if they are in the same channel as you.