I suppose you could get your Japanese passport stamped every time you left and came back, but then the timelines might be an issue. Also, whether or not the customs official even would.
Now suppose you flew in and out of Canada every time, using your US passport to enter and Japanese to exit....
As long as your were diligent with keeping track of it, it would look like you were in Canada to the Japanese government.
Funny story, about keeping these things straight. Going back to Japan after travel with my infant son to Guam. Wife and my son go through the Japanese immigration while I have to go through the foreign nationals line, wife had accidentally handed my my son's passport. The look on the official's face as he stared at the baby photo was great, I told him "Sorry, I didn't shave so I look a little different."
It and Japan were our top two locations for settling when I left the military, but life has a way of ignoring what you want and we ended up elsewhere for difficult reasons.
I lived in Japan for several years, love living there but don't like working there. I am back in the states on the east coast but I hope I can retire over there.
If you only ever travel between the USA and Japan, they wouldn't know anything since the USA doesn't stamp your passport anyways.