All my friends were Japanese until middle school, and my social life was dominated by Japanese. Definitely didn't fit into American elementary school at all and was a problem student.
But then I stopped going to Japanese school in HS, and then figured out how to excel in American school, met likeminded friends for the first time, etc.
Then I went to work at a Japanese megacorp in my early 20's.
It's a strange mix of time spent using the language.
In a vacuum, it's a net positive to be able to understand an additional language very well. But I've felt that my life would be easier in Japan if I were "more foreign", since I wouldn't be subjected to the expectations of a domestic person while actually internally being a foreigner.
Also I'm convinced that had I spent the resources that would put towards Japanese into English, I would have become much more proficient at English earlier in my life. That would have translated into better verbal test scores, and may have meant I wouldn't have had such a hard time in my younger years in school. Then again plenty of solo-English speakers of the shy nerd bretheren have problems at school of not fitting in, so this isn't only a language issue.
Honestly it's just a grass is greener type of situation. It's brought me great advantages (I wouldn't be doing my current company without this advantage, and I genuinely think the language is a beautiful and sonorous one, and the cultural traditions are admirable), but it's also held me back in many ways compared to my 100% American English peers as well.