"And nope, Signal doesn't make the cut for the above reasons because it exposes your phone number to everyone else"

This is being worked on.

The thing is you're mixing two threat models. One is a creepy dude who will give you nightly calls if they learn your phone number. The other is a state actor who will hack the server and track you based on your IP-address if no phone number is being used otherwise: hence the enumeration attacks won't matter. You can't escape state actors looking at your metadata with Wire, Element or Signal. For that you want an Onion Service based system like Briar, Cwtch, Ricochet, or TFC.

For the creepy people not having to hand out your phone number is a nicety, but it's not at all hard to block a phone number either, it works just like any other app's blacklist: just add the number and be done with it.