Lack of marketing/branding and UX that's harder for less technically inclined people to understand. The majority of people want to be able to install the app, enter their phone number then see which of their contacts are using the app. Telegram does this very well, Signal does this pretty well and Element does a pretty poor job.

In fact, I have joined a couple of IRC channels in the past and now there are literally hundreds of random people who I don't know in my contact list. Maybe there's a configuration option to fix this but who wants to spend time configuring their IM app.