It's because Signal chose to make the phone number their main identifier, not some random username or email.
(I can't immediately find their rationale for that decision, but I think it had to do with making the service available to those millions that have a cell phone but no email, and usernames make it hard to auto-populate contacts whereas phone numbers can be cross-referenced with your contact list)
Signal uses phone numbers because Signal users already have contact lists outside of Signal. By piggybacking on phone contacts, Signal can avoid storing metadata about who's sending encrypted messages to whom.
Compare that with Wire, which is a fine system, but which also effectively stores a log of every pair of users that have spoken to each other on the platform, in a database, constantly available.
Not having that database is the win for phone number identifiers.
Meanwhile, while the loss from that decision is painfully felt by people who communicate on platforms like HN, it's hardly felt at all by ordinary users, who already communicate primarily on platforms that use phone numbers as identifiers. WhatsApp, the most popular messenger in the world, was created as a pin-compatible replacement for SMS.
Signal's decision here is not the decision I would have made, because I loathe phone numbers (and, for that matter, messaging people on my phone). But it was a smart, principled decision, and almost certainly the right one; I'd decide otherwise because protecting the most people in the most effective way might be my stated preference, but it wouldn't be the preference my own actions would reveal.
I get that using the contact-lists, and the constant re-uploading of contact-lists, is great for Signal's bootstrapping. Anchoring IDs to phone numbers might also work as an abuse throttle.
But I don't see how it minimizes metadata. Can you clarify?
But Signal could still be keeping a database log of everyone who's actually ever sent a message to anyone else. (To the extent anyone's using 'sealed sender', it'd be harder – but apps like Wire could do something equivalent to 'sealed sender', too.) And that database is way more valuable to many attackers if it's keyed by the phone-numbers Signal has, compared to the aliases other systems have.
And to the extent the Signal client wants to keep re-uploading my entire contact-list, even a one-time, temporary compromise of their SGX-based system would reveal all my phone contacts.