I see my phone as a way to connect with people rather than a way to simply communicate with people. The fact that I might have to listen to people pause and form their thoughts is a positive of voice notes, not a negative. I don't ask my friends to provide transcripts before I meet with them in-person to make sure my time isn't wasted when we speak.
I understand that if you are receiving work-related voice notes that are better suited for text then that would be frustrating, but hearing my friends speak is so much more preferable to just reading their words on the page.