You've hit upon the limits of Signal and other messengers.
The version on app stores is not a closed source version. However it is a binary, and there might be questions on build reproducibility. I do not know the answer to this nor the answer to your MRENCLAVE question.
Virality is because of their philosophy - their first goal is to end mass surveillance, not provide custom software for preventing individual surveillance. The quicker everyone in the world is using E2E, not only is there less mass surveillance, contact discovery leaks zero additional information at that point.
Concerns with SGX are real - but it remains the state-of-the-art - your criticism assumes that the competitors do any better, at this point they do not. They have traditional backends or are as flawed. Signal is doing the hard work on researching solutions at this point, the others are not as close.
> Signal could generate a long enough key locally, and if you want to add another signal user, the client could send it automatically to a contact through SMS. The client on the other side could automatically read the key through SMS and add the contact. Or the user could manually send the key through any mechanism they wanted.
This is what happens when contacts verify each other through QR codes on Signal. But this mechanism does not solve your problem nor are you solving the problem Signal wanted to solve - minimizing data on servers. Even with keys, servers still has to route messages, and with your solution they'll have to maintain a user database.