Is this attitude a holdover from encouraging small children to read for themselves? I don't know if this is quite the right framing but are you... hearing-illterate to some degree?
Some voice actors definitely add a layer of interpretation, and some are distractingly bad, but otherwise it could not be any more of an equal experience absorbing a book through my eyes or my ears. Unless there was a need to physically interact with the book, quickly flipping between sections or keeping notes in the margin, which I rarely do.
"I didn't really get the full experience until I read it on the page instead of listening" is an absurd statement to me, but maybe people's brains work differently.
There are things missing from text that can be conveyed over audio or with body language in person certainly so those experiences can't be directly translated from real life to the page.