From what I can see, they're doing everything that they can to eliminate sending anything to servers. They recently launched private contact discovery which is really interesting.

https://signal.org/blog/private-contact-discovery/

Moxie has said repeatedly that they're working to do better on this front, but it's not without its technological challenges. He's written repeatedly about this and has also called for anyone willing to try to make it happen and offered to help them. Just one example springs to mind:

https://news.ycombinator.com/item?id=12883410

And federation is possible over XMPP with Signal:

https://signal.org/blog/the-ecosystem-is-moving/