I didn't speak Japanese when I first moved there. In larger cities (especially Tokyo), there are plenty of people who speak English, so it's generally not a huge problem (although you'll find things easier if you do speak the language).

The biggest issues are crowding and racism. Don't expect to get into any role of importance, and don't freak out when you hear "gaijin dame" (no foreigners allowed) in some places, or people sudenly moving to seats and tables further from you. On the other side of the coin, you won't be expected to understand how things work, so they'll be very forgiving (in a condescending way) of social gaffes and the strange way you act (after all, you're a foreigner; they all act that way). Many rules won't apply to you, unless you make an effort to have them apply to you (don't do this). You'll make few (if any) friends of the same gender. You will alway be an outsider. If you can live with this, you'll have a fun and interesting time there.

Edit: Also, make friends with some oldtimers in the expat community (as long as they're not the jaded kind). They are a treasure trove of information and help.

Edit 2: Don't learn Japanese from someone of the opposite gender (unless they're a trained language teacher). You'll sound ridiculous when you speak otherwise, because men and women use very different words and turns of phrases in many cases (nothing worse than hearing a man say "atashi" or the up-pitch "ne" unless he's in drag). Reading Japanese comic books can be helpful here because they tend towards hyperbole, which makes it easier to pick out the differences. I've found that female artists tend to play with language more than male artists.

Edit 3: DO NOT BREAK THE LAW. Getting arrested in Japan is a hellish experience, and prison is even worse. Their justice system is still quite medieval. Quoting a friend of mine: "Japan is the most advanced 3rd world country I've seen."