Matrix defines a sort of end-to-end encryption, but the ends are homeservers and clients. [Some people are saying not: that homeservers don't see plaintext of E2EE traffic.]
There is talk about self-hosting in the client, but I don't know if it works yet, or ever will. Lack of encryption-at-rest, wherever it is that messages live, seems like a stupendous implementation design flaw, and makes me question all the project's other choices.
If, in fact, messages are, or can now be, stored securely, I would welcome correction. Likewise, if client-side hosting works now, or message-store migration, or a stable address despite such a migration, or any effort at securing metadata. I have not kept up since abandoning Matrix, but still want a viable alternative to Signal.
The Matrix protocol is extremely complex and getting more complex with great speed as they try to get to feature parity with Facebook and Twitter, making it hard to believe one will ever be able to trust it, E2EE or no.
Will we need to start all over again? A rigidly layered system, with a provably secure basis, probably in a single, sandboxed server talked to by all clients and gateways, with services built on top, seems needed if we want both security and features.
As it is, it seems like clients -- i.e. application services -- run in the same address space with what should be secure message transport, necessarily compromising all security with each bug added.