For example, let's say the centralized service has a virtual number of 123-456-7890. If someone with the number 444-444-4444 calls it, it routes to my actual number. If someone with the number 555-555-5555 calls it, it routes to someone else's number. If anyone else calls it, it just doesn't work. If I remove the number from my account, when 444-444-4444 calls it, it not longer works.
In my mind, this is similar to having a virtual card that rejects the transaction if it doesn't match a specific merchant/category. Since phone numbers are more limited, you'd have to have shared virtual numbers, whereas virtual cards are 100% unique.