Not at all, I have a bunch of friends on my private server and getting someone an XMPP client and account takes less than a minute. No phone number or email required.
Sure, everyone out there isn't going to be on my server - but for the close friends I care about - it's been flawless.
I decided to delete all proprietary messaging apps and use XMPP exclusively in Janurary this year. It took some time to explain but now most of my friends and family can now be reached via XMPP.