From my best interpretation of your comment you seem to think VoIP is far more "carrier peer to carrier peer" than it is in practice. As a value added telecom provider I have a few upstream carriers but it's not as if I receive calls from
every carrier. Just like the Internet, my upstream carriers are my access points to the PSTN, so all of my SIP invites come from either my customers or my know upstream carriers. Those carriers are effectively my ISPs of the PSTN. I chose them and I pay them for service.
Distrusting one of them and the calls that come from them is something of a non-sequitur since you chose them and intentionally became one of their customers and pay them to receive those calls.
TL;DR This isn't like email or IP traffic where you're interacting with other IPs in a purely untrusted network model. Merely analyzing the previous hop doesn't tell us anything valuable.