If I remember correctly, gaijin is just how you say 'foreigner' in standard Japanese. The rude slang is jingai.

And I also have the experience of people really appreciating it if you actually speak and read Japanese. Which makes sense, I can easily imagine it being a relief to find that you can just speak with someone normally instead of having to struggle with this absolutely bonkers weird language that one may be only vaguely familiar with.

I can imagine that people in, say, the United States wouldn't be very happy if I went around and only spoke Dutch.