It is hard not to become a train enthusiast in Japan, especially when armed with a rail pass. I would essentially wake up in the morning, decide where to go, within the hour I had tickets and was on a train. Book a cheap hotel on the train ride, enjoy the new city until I felt like another train ride. Effortless and perfect for the unplanning wanderer.
For the bus I mentioned though, I just used Google Maps/GPS on my phone and guesstimated where to pull the stop request cord, etc.
For communicating with people, Google Translate works decently well. Obviously knowing Japanese would be ideal, and you should learn a few phrases, but it's ok to use Google as needed.
Can't speak for super rural areas, though.
The rail pass gives you a lot of freedom since it allows you to hop on a Shinkanen without booking your seat in advance.
Though I ended up learning some phrases while there so I could be more polite. The restaurant protocol really helped, and "sorry", "excuse me", "thank you", "hello", "goodbye" were all helpful.
The trouble with using only some Japanese is you will get a reply in Japanese and not understand it, so sometimes it's best just to be up front that you don't know it.
The trains themselves are effortless once you do your first one. You quickly learn the floor markings on the platform, and you can't mess it up too badly because if you miss one, the next one isn't far behind.