Sure, going from recognizing letters to actually reading is what I meant by learning language. If you have some degree of familiarity with a language, you're no longer looking at individual letters and pattern matching them with symbols in your head. You're able to take in a word or multiple words at a time.

And at least some of the English/Japanese difficulty is due to fact that you're also changing systems of writing, going from an alphabet to a pictographical system. Comparing that with English to Spanish is almost unfair because of the fact that so many English words of French and Latin origin have very similar cognates in Spanish. I'd imagine you'd have similar troubles if you tried to learn Hindi/Devanagari for instance. Devanagari is an abugida, so you'd need to learn all of the letters in the alphabet and a whole bunch of "modifications" to the consonants that denote adjoining vowels.