Why does Signal scare you? All of their code is open source[1] and has had multiple public audits[2].
The alternative (using Telegram's homegrown and dubious cryptography and lack of actually encrypted group messages) is significantly worse. Here's a nice summary of the issues: https://security.stackexchange.com/a/49802/43733
IP address is much easier to hide, for example by using a VPN.