I can totally relate. Reading lots of english makes very little to your listening skills.
I mostly "solved" it by listening lots and lots of podcasts, but I can still remember googling the word "ossom" which a podcast host repeated every two or three sentences. I was puzzled to discover that he was actually saying "awesome", a word I definitely knew and I was familiar with.