Client devices have many routes of compromise available, it's very possible that the data exfilaration is occuring there rather than server side.
You'd think the backers of Matrix (largely European Governments) or the privacy focused userbase of XMPP would go beyond forklifting in the same double rachet library that Signal and ilk use, but apparently they all think it's the best available option.