There is also something else I notice: some people will decide they will not understand you because you look/are foreign no matter how hard you try or even if you don't make not many mistakes. There is a famous video about that[1] but this is not limited to Japan, I also saw it in France too with foreigners trying to speak French.
Also even in a cultural area there can be wide differences. Almost all Mandarin speakers I interact with (be they Taiwanese, mainlanders or huaqiao) seemed quite happy to talk their language with me. The Japanese not so much. I even sometimes hide my Japanese language ability and play it dumb to fit more in a group. And I plan not to expose a too great ability the day I got it unless necessary precisely too avoid being judged to harsh by the local standards.