Spent four years in Japan. Got stopped on a regular basis by cops in Tokyo to be inspected for drugs and weapons. Even twice within the span of a kilometer once. Anecdotal, but there you go.
Thing is, I know plenty of foreigners who have a similar experience to your own, so I know you're not wrong. I don't know what I do differently, but Japanese cops do not bother me at all.
wrt moving back, I will be leaving next year due to the job market here, so there's a data point for you. It's not impossible to find work, but if you're in tech and coming from the US be prepared for a bit of a shock.