Plus, XMPP can be used for great amount of stuff non-IM related, while Matrix seems very focused around its IRC-like use case, where XMPP doesn't really improve much over IRC.
I don't really agree here. While Riot is focused on a IM use of Matrix, Matrix itself is open enough to be usable in any use case requiring a payload to travel from A to B. You could think of Matrix use in IoT, social networks, forums, etc. Some people even made a working PoC of a blog using Matrix, and I heard some others are also working on building a system to bypass information censorship using Matrix as its only back-end.
I don't know XMPP enough to state which spec is wider than the other, but what I know is that Matrix isn't limited by what Riot or other Matrix-based IM clients can do.
GP already stressed it out:
> Matrix's main data primitive is synchronising conversation history within a room - not message passing.
So Matrix might be well suited for things like blogs, but XMPP will be certainly better for push notifications, for instance. However, you can easily build Matrix-like primitives on top of XMPP, while the other way around will be less flexible.