Now suppose you flew in and out of Canada every time, using your US passport to enter and Japanese to exit....
As long as your were diligent with keeping track of it, it would look like you were in Canada to the Japanese government.
Funny story, about keeping these things straight. Going back to Japan after travel with my infant son to Guam. Wife and my son go through the Japanese immigration while I have to go through the foreign nationals line, wife had accidentally handed my my son's passport. The look on the official's face as he stared at the baby photo was great, I told him "Sorry, I didn't shave so I look a little different."