I share the view that end-to-end encryption (E2EE) provides strong security guarantees only if the clients used to exchange data/messages are fully under control of the end users. All it would take Signal to undermine their E2EE is a single update of their app that exfiltrates all chat messages (not saying that I think they would ever do that). Same story for WhatsApp. In my opinion, a necessary ingredient for secure E2EE is that you don't have to trust the central infrastructure operator, at all. And that just doesn't work if the provider controls the client software.

Another problem is that almost all E2EE messengers can still be trivially man-in-the-middled (MITM) by the infrastructure operator, as there's no out-of-band verification of keys. The central server can just replace client keys in transit and decrypt all the data exchanged between two clients. This can only be detected by clients comparing their public keys over an out-of-band channel. Most E2EE messengers simply disregard this risk [1], while some provide functionality for out-of-band key verification (e.g. Threema). Always struck me as a bit odd as it's like using self-signed TLS certificates for your server, which leads to a security exception in all modern browsers but somehow seems to be fine for E2EE messengers. Keybase tried to solve this problem but unfortunately they got acquired by Zoom so I doubt they will continue working on that.

[1] https://signal.org/blog/there-is-no-whatsapp-backdoor/