Compared to Telegram, it feels like using a laggy MSN Messenger. The experience, both client and server-wise, just feels very unpolished. It's no single big thing, it's more like death by a thousand cuts.
I was bullish on Matrix because it's so extensible, but in the end I realized that only the default client experience matters as that's the one everyone will be using. And it just isn't there yet. In the end, all the group chats I was in migrated to Discord or Telegram, so I had no more reason to use it...