Seems the other way to me and I would guess you’ve never tried it for any length of time. I get 4x as many junk calls as I get legit calls. If I don’t recognize your number, I send you to VM. My parents are elderly but they leave a message or send a text if I don’t answer. I add my doctors offices, city dispatcher, and similar numbers to my phone as soon as I learn them. It’s rarely a problem to hear a message and call back a few minutes later (except when they also don’t answer, in which case I leave a message and now have their number in my phone). Even works for large multi-location offices, who often show the same number no matter which is calling. My (adult) kids have been involved in emergencies and I see “City Dispatch” come up and I answer.
It’s just not a problem.