Even that wouldn't be necessary. Here's an idea – when receiving a call, the receiving network could separately connect to the claimed number and verify its authenticity. No match? Reject the caller. The owner of a number (and thus its cryptographic identity) could then delegate other agents to use its keys, its phone number, and most importantly, its reputation. Cryptography has come a long way since the days of rotary phones. The entire handshake could happen in under a second. The majority of calls these days are VoIP anyway. In fact, maybe we only implement this for VoIP and finally force the world to upgrade.

I remember the days before HTTPS was widespread. I watched as Google and other players systematically adjusted incentives, slowly but surely over many years, to reach where we are today: 90% of all web traffic is encrypted.[1] There's no reason the same couldn't happen with phone calls, other than perverse incentives.

Someone get me in a room with Ajit Pai and the CTOs of the major US carriers, I could have a system in place in five years, tops. Unfortunately there's no incentive for them to fix this, because (1) upgrading infrastructure costs money, and (2) carriers large and small are making BANK off these robocalls. ITSPs do typically charge to route calls (similar to internet peering), and robocallers are happy to pay because they're making so much money.

[1]: https://news.ycombinator.com/item?id=21421195