Why? The main chat server matrix.org has a child porn/CSAM 'problem'. Due to lack of moderation in many of those rooms, along with protocol problems, these sorts of CSAM spammers can do an hours long image campaign of stuff that's illegal to even have. Theres nothing quite like waking up to a post every 10 seconds of felonies in a cybersecurity or Linux chat, and the summary clean and get the hell out of those rooms.
Banning also doesn't work, due to distributed nature of rooms. You can be banned from matrix.org room but connect through a different server, and they can still spam users.
If you do want to be on Matrix, I would recommend a few changes.
1. Don't stay on matrix.org chatrooms. They are the worst hit and slow to resolve
2. Disable image preloading and downloading.
3. If you have private servers and rooms for friends, its the best.