Japan desperately needs immigration and desperately doesn't want any.
Of course it's not just about visas. If there is a way and a reason to immigrate, people will do so, even illegally. Japan can't even benefit from illegal immigration because its geographic properties isolate it so well.
This is the exact reason they desperately don't want immigration.
So, I do not live in Japan, but I've visited multiple times now. I don't speak Japanese more than a few words here and there and mainly get by usually with Google Translate.
I've had a not-too-difficult time making friends (who I even stay in touch with from here in the US) by finding people there who participate in the same hobbies (car culture and baseball, specifically). Generally, most people I've met in those communities are very friendly. Not everyone enjoys the Google Translate back and forth but many still do.
Anyway, like I said before, I don't live there so it may be different as a long-term resident. But I just feel like based on my experience so far, the "way in" may just be a matter of finding people with similar interests, much like I would here in the US.
Most of us quickly made casual friends as you described, but are yet to make many close friends. As for integrating into society, most of us found society quick to help and welcome us, but struggled to move on from "the foreigner that needs help and teaching" to "the person with ideas and opinions as valid as an equivalent Japanese person", this is irrespective of Japanese ability.
Perhaps that is why many people says it's difficult to integrate, what we immigrants want from integration and what Japanese society wants from integrated immigrants is fundamentaly different.
As a person who lived in many countries I felt the same in almost every english speaking countries, they could be welcoming but making friends isn't that easy, maybe easier in NZ and OZ than the rest. In Latin countries I felt it was a lot easier, especially coming from one myself.
This isn't true in my experience. The wages for IT work are much better in the US, but that's it. Tech salaries in Europe are just as bad as in Japan, AFAICT, but housing costs in Europe are worse.
>Aside from the language issue, it is also difficult to integrate socially if you are a foreigner.
This is true everywhere. Talk to anyone about their experiences of emigrating to Germany, for instance. Or someplace in middle America like Iowa. From my experiences with immigrant friends in the US, their social circles are mostly other immigrants or expats from their home countries.
Aside from skilled workers, they've eased up a lot on the unskilled workers, so there's many from SE Asian nations like Philippines, Thailand, and Vietnam working in service positions. Of course, westerners who come here probably don't notice this because they think all Asians look alike...
What I have noticed is internet commenters who have never even been to Japan, let alone lived here, talking about the "casual racism" that they think is so common here just because they read someone else on the internet make that claim. I see this in probably every discussion about Japan either here or especially on Reddit.
Shortage of labor is a definite problem in Japan, lack of progress or flexibility is another one. Both can be helped by immigration.