This was far outside of Tokyo and seeing a 'gaijin' was a genuinely rare occurrence. More than anything the word seems to fit a real communicative need. "I had an interaction with someone that was different from the norm because they did not look Japanese" is quite a mouthful and invoking the word 'gaijin' starts to happen quite naturally.
My biggest problem with the word is that it is used sloppily. It conflates race, culture, language and nationality and assumes a kind of all or nothing approach to them. My black student who was born and raised in Japan fulfilled 3 out of those 4 categories but would be considered gaijin everywhere he went.