Well I'm east Asian so I wont stick out like a white person. But no, you wont have any hard time. Whether you will be accepted as Japanese is another story.