I'm sorry, that's simply not true. At least for me. It may be true for you.
From the things you've said, I suspect you're a word-at-a-time reader, treating words as a sequential symbol stream, processing them as if they were speech. This is just one of the several different styles of reading. Others exist.
I am, as I mentioned above, a phrase-at-a-time reader. I take in multiple symbols at a time, and not necessarily in sequence. If I had to think through the audio of a phrase, this simply wouldn't work. It also means that I would be unable to read symbols that didn't have an audio equivalent.
Right now I'm working with Smalltalk. One of its operators is ~~. How is this pronounced? Don't know, don't care. It's just ~~. When I perceive it, I don't perceive 'tilde tilde'; it's a ~~.
(Also, if you aren't reading Scotland's worst poet aloud, you are wasting him.)