Actually Japanese know it is hard, and focus on it like madmen. A top Nintendo DS game for adults in Japan is a Kanji re-training game.
In any case, Kanji is so visual - until I learned it many words didn't make sense to me. Knowing how to say it, but not understanding the real-underlying meaning is not fun.
As far as intonation, Japanese is one of the easiest languages with regards to that. Once you get the basics it is simple, but still you must talk with others (Japanese club, co-workers, neighbors, skype, etc) and listen to real Japanese (anime, news, etc.)