Huh, I'm the opposite, but I think it depends on the subject matter and how much you're reading. I have read about 70+ fantasy and science fiction audiobooks in the last year, but only each one once, and remember most details of all the books. Some characters from some series I've no doubt forgot their names, but soon as I listen to passage from the series my memory is brought back and I remember the details.
Non-fiction on the other hand I don't do too well with audio books. e.g. listening to an audio book on programming is never going to happen for me.
I think I'm the opposite. When I was in school I was able to learn things much faster by attending the lecture compared to reading on my own.
my bet is if you test yourself on a book you listened to two years ago you will not remember 99% of it, and if you read it it's probably more like 95% percent of it.
then again, everyone's mind is different, so maybe you retain voice more than audio