For what it's worth, I've been to Japan twice now for a total of one month of time, don't know any Japanese beyond your typical easy phrases/words, and got by fine with Google Translate. I also was able to make several friends, many of whom can't really speak much English (probably on the same level as my incredibly basic phrase/wordset of Japanese). Not only that, but once I made those friends they took me to places/restaurants that tourists wouldn't know of/go to and only locals would.
Of course on the other hand there were some areas where I would be by myself and it was clear that unless I could speak Japanese I wouldn't really be welcome- and that's alright too.
But my main point is that it is certainly possible to not know Japanese and still see quite a lot of Japan; I suppose it just takes some effort and outgoingness to try communicating with the locals.