I recently did the same thing, as 95% of my incoming call volume in a week was spam calls. It's been great. The friction I feel is when interacting with ephemeral contacts like contractors, etc. I've had to try to be diligent about adding them as contacts if I expect a call back, or hoping they leave a voicemail.

It's sad there really isn't much you can do about it. I tried do-not-call lists, answering and telling them to stop calling me, reporting them - all was apparently a waste of time.