There are countries in the world where homeless are pretty good hidden, by means of extreme expensive welfare, or are moved away from big cities, or at least touristic centers.
I have no idea in Japan. As I was there I saw extremely poor people (deduced from cloths and lack of hygiene) I doubt they had an own house. Even worst, I saw middle-class neighborhoods that I would associate with a favela in Brazil (albeit very clean and organized, each flat was smaller than a space in Rio.