Also the reason I have 80+ contacts in Signal, but 1 (one) in Matrix. Federation just doesn't work for the average person in practice. Email being the large exception because it's been around for so long.
what does cause friction is if clients expect the users to enter their own server details beyond their email address.
i find that very annoying in irc clients for example, where i have to know the details for the network to connect to. jabber/xmpp clients generally did a bit better. and i don't know how well matrix is doing it. but making this easy for the user is a matter of interface design.
there is nothing inherent in federation that makes this hard for users. the only thing is having to choose a server. but for my family for example i would choose the same server as myself, and i'd want to send them a link to sign up. that link could then provide a way to open the messaging client with proper settings in place.
if it is more complicated now, then that's a problem, but one that can be fixed