I've been studying Japanese as my fourth language for over 2 years, and I have to disagree with both you and the OP. If you didn't find Japanese to be very gramatically challenging or interesting, you unfortunately probably didn't get into any keigo, which contains grammar of a mindbogglingly high level of complexity that is probably only surpassed by a similar system in Korean, since Japanese grammar was artificially simplified during the Meiji Restoration in order to make a simpler national dialect that all Japanese could easily learn/use. Nevertheless, the situational shifts in Japanese grammar based upon the relationship between the speaker and the listener are very difficult to become accustomed to, even for native speakers.