For metadata you just have to trust them. Sealed sender doesn't solve that[1], even if it's better than nothing. It's also better than nothing to require no phone number in the first place (Wire), or not to require a payment (like Threema does) which is roughly as hard to make anonymous as getting an anonymous phone number. But either way, they can track everything you send, it's a matter of wanting to. The only way to avoid that is by not sending personal data to semi-/untrusted parties at all (Matrix).
Since centralized services are currently also the convenient ones and neither Wire nor Signal show any sign of wanting to use a decentralized protocol, those that can't be bothered to use inconvenient services have to trust that their centralized service is ethical about what they do with your data.
The privacy differences are very minimal by comparison when they're all missing one basic feature or other like message editing (Signal), a desktop client (Threema[2]), cross-chat searching (Wire), searching a chat at all (Element), any usable encryption (Telegram), etc. If you're going to use an end-to-end encrypted messenger where the server can't read any contents, the privacy differences are just not that large when you trust them all equally. The only thing that you can objectively compare is what the client sends, since that's something they can minimize and you can actually check.
[1] "clients derive a 96-bit delivery token from their profile key and register it with the service. The service requires clients to prove knowledge of the delivery token for a user in order to transmit “sealed sender” messages to that user". It's a bit opaque so in regular English: my Signal client sends something like deliveryKey = H(profileKey, currentTime) to Signal, where my profileKey is something only my contacts and I know. Great, so my contacts only specify the deliveryKey and not who's sending, so you send anonymously! But wait, those contacts connect to Signal with an IP address and are doing other things like updating their profile or registering their delivery keys for their account from that same IP address. 1+1=2 and you know who is sending messages to whom.
[2] They have it, but your phone plays Chinese Whispers with your computer and every time your laptop or phone reconnects to wifi/mobile data you need to open the app on your phone and navigate two menus to re-enable it.