While it's worrying that Matrix uses the layer 7 HTTP (does it make it "layer 8"?) compared to XMPP's potentially more efficient use of layer 4 TCP, it's still better than the closed alternatives...
(And JSON > XML !)
You are going to be more suspicious if you are using $protocol-specific-for-encrypted-chat than if you are just browing the web.
> Bits are bits
I agree, but this does not stop my ISP from throttling my SSH speed.
There are also replay attacks and probably many other issues I can't think about. Maybe some IM protocol already deals with that.