From a remembering the tune perspective, I have the same issues, but I think it's more related to not applying musical lexicon and hearing skills the same way: you remember poetry or a paragraph of text because you remember the ideas and how to go from one to the other, if you are a musician and have something in your head and start thinking along the lines of "this is using a lydian mode, the progression is ii IV V I then it modulates to the relative minor and switches to dorian, also the theme is going down in thirds for two bars, then it will stay on the chord root for one and move to the dominant 7th" you are going to remember it a lot more easily than just by remembering the melody itself
It would be like comparing how easily you can remember poetry in English vs poetry in, say, Russian, where you only have the "sounds of the words" in your head to remember, but you don't have the syntax or the meanings to help you as well.