Reading and listening use different types of memories so that's not enough to learn it. You have to actively recall to learn the language.
Also, keep in mind that there is no such thing as permanent fluency. You have to continue to practice it if you want to always be fluent. Even native speakers will lose their fluency in a language if they don't speak it. I've seen that a lot with kids that learned their parent's language at home but decide that the will only speak English once they start school. They eventually lose their fluency. They might understand their original language but will definitely have trouble speakng it.
I three years of German in high school, and one semester in college. I never got that "der", "die", "das" stuff figured out until I got interested in reading German. At that point I found that you have to know how to parse grammatical agreement in German if you want to know who did what to who.
I spent a year working in Germany later on and at one point I realized that I could go to to the bank and take out money and do many similar tasks.
Since then I've gotten into Japanese animation which I watch with subtitles. At this point I figure I've spent as much time listening to Japanese as a child does in the language acquisition phase. Sometimes I forget to turn the subtitles on and it takes me a while to remember.