I've tried, really hard. I did 4 years of french, 2 years of spanish, 4 semesters of Arabic, 2 semesters of german, 1 Berlitz tutor session for 3 months in Russian and 3 months of tutoring in Farsi. Most of my grades were AWFUL and I barely qualified for anything. I really tried, but I just don't have the medium/long term memory for language. It also doesn't help that I live in the US.
There were people who still raced around me -- I'm not gifted at languages, by any means -- but for the first time in my life I'm actually modestly capable in a second language. I continue to study now, and it's still accumulating, however slowly.
After this experience, I believe that most people can learn a second language, but they don't because the barriers to entry are insanely high. If you really want to learn a language to a conversational level of fluency, you either have to be truly gifted (5% of people) have to want it so much that you'll immerse yourself in it 24/7 (the rest of us).
> I did 4 years of french, 2 years of spanish, 4 semesters of Arabic, 2 semesters of german, 1 Berlitz tutor session for 3 months in Russian and 3 months of tutoring in Farsi
It took me about 3 months of daily immersion in Japanese life (plus the 4 hours of daily school, of course) just to get to a point where my ears could hear words reliably. If you translate this to coursework...it's got to be 1+ years of college-level study.